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Preface 


Lk writing this book, I was guided by my long-standing experience and interest in teaching 
discrete mathematics. For the student, my purpose was to present material in a precise, 
readable manner, with the concepts and techniques of discrete mathematics clearly presented 
and demonstrated. My goal was to show the relevance and practicality of discrete mathematics 
to students, who are often skeptical. I wanted to give students studying computer science all of 
the mathematical foundations they need for their future studies. I wanted to give mathematics 
students an understanding of important mathematical concepts together with a sense of why 
these concepts are important for applications. And most importantly, I wanted to accomplish 
these goals without watering down the material. 

For the instructor, my purpose was to design a flexible, comprehensive teaching tool using 
proven pedagogical techniques in mathematics. I wanted to provide instructors with a package 
of materials that they could use to teach discrete mathematics effectively and efficiently in the 
most appropriate manner for their particular set of students. I hope that I have achieved these 
goals. 

Ihave been extremely gratified by the tremendous success of this text. The many improve- 
ments in the seventh edition have been made possible by the feedback and suggestions of a large 
number of instructors and students at many of the more than 600 North American schools, and 
at any many universities in parts of the world, where this book has been successfully used. 

This text is designed for a one- or two-term introductory discrete mathematics course taken 
by students in a wide variety of majors, including mathematics, computer science, and engineer- 
ing. College algebra is the only explicit prerequisite, although a certain degree of mathematical 
maturity is needed to study discrete mathematics in a meaningful way. This book has been de- 
signed to meet the needs of almost all types of introductory discrete mathematics courses. It is 
highly flexible and extremely comprehensive. The book is designed not only to be a successful 
textbook, but also to serve as valuable resource students can consult throughout their studies 
and professional life. 


Goals of a Discrete Mathematics Course 


A discrete mathematics course has more than one purpose. Students should learn a particular 
set of mathematical facts and how to apply them; more importantly, such a course should teach 
students how to think logically and mathematically. To achieve these goals, this text stresses 
mathematical reasoning and the different ways problems are solved. Five important themes 
are interwoven in this text: mathematical reasoning, combinatorial analysis, discrete structures, 
algorithmic thinking, and applications and modeling. A successful discrete mathematics course 
should carefully blend and balance all five themes. 


|. Mathematical Reasoning: Students must understand mathematical reasoning in order to 
read, comprehend, and construct mathematical arguments. This text starts with a discussion 
of mathematical logic, which serves as the foundation for the subsequent discussions of 
methods of proof. Both the science and the art of constructing proofs are addressed. The 
technique of mathematical induction is stressed through many different types of examples 
of such proofs and a careful explanation of why mathematical induction is a valid proof 
technique. 
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2. Combinatorial Analysis: An important problem-solving skill is the ability to count or enu- 
merate objects. The discussion of enumeration in this book begins with the basic techniques 
of counting. The stress is on performing combinatorial analysis to solve counting problems 
and analyze algorithms, not on applying formulae. 


3. Discrete Structures: A course in discrete mathematics should teach students how to work 
with discrete structures, which are the abstract mathematical structures used to represent 
discrete objects and relationships between these objects. These discrete structures include 
sets, permutations, relations, graphs, trees, and finite-state machines. 


4. Algorithmic Thinking: Certain classes of problems are solved by the specification of an 
algorithm. After an algorithm has been described, a computer program can be constructed 
implementing it. The mathematical portions of this activity, which include the specification 
of the algorithm, the verification that it works properly, and the analysis of the computer 
memory and time required to perform it, are all covered in this text. Algorithms are described 
using both English and an easily understood form of pseudocode. 


5. Applications and Modeling: Discrete mathematics has applications to almost every conceiv- 
able area of study. There are many applications to computer science and data networking 
in this text, as well as applications to such diverse areas as chemistry, biology, linguistics, 
geography, business, and the Internet. These applications are natural and important uses of 
discrete mathematics and are not contrived. Modeling with discrete mathematics is an ex- 
tremely important problem-solving skill, which students have the opportunity to develop by 
constructing their own models in some of the exercises. 


Changes in the Seventh Edition 


Although the sixth edition has been an extremely effective text, many instructors, including 
longtime users, have requested changes designed to make this book more effective. I have 
devoted a significant amount of time and energy to satisfy their requests and I have worked hard 
to find my own ways to make the book more effective and more compelling to students. 

The seventh edition is a major revision, with changes based on input from more than 40 
formal reviewers, feedback from students and instructors, and author insights. The result is a 
new edition that offers an improved organization of topics making the book a more effective 
teaching tool. Substantial enhancements to the material devoted to logic, algorithms, number 
theory, and graph theory make this book more flexible and comprehensive. Numerous changes 
in the seventh edition have been designed to help students more easily learn the material. 
Additional explanations and examples have been added to clarify material where students often 
have difficulty. New exercises, both routine and challenging, have been added. Highly relevant 
applications, including many related to the Internet, to computer science, and to mathematical 
biology, have been added. The companion website has benefited from extensive development 
activity and now provides tools students can use to master key concepts and explore the world 
of discrete mathematics, and many new tools under development will be released in the year 
following publication of this book. 


I hope that instructors will closely examine this new edition to discover how it might meet 
their needs. Although it is impractical to list all the changes in this edition, a brief list that 
highlights some key changes, listed by the benefits they provide, may be useful. 


More Flexible Organization 


= Applications of propositional logic are found in a new dedicated section, which briefly 
introduces logic circuits. 


® Recurrence relations are now covered in Chapter 2. 


™ Expanded coverage of countability is now found in a dedicated section in Chapter 2. 
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® Separate chapters now provide expanded coverage of algorithms (Chapter 3) and number 
theory and cryptography (Chapter 4). 


® More second and third level heads have been used to break sections into smaller coherent 
parts. 


Tools for Easier Learning 


® Difficult discussions and proofs have been marked with the famous Bourbaki “dangerous 
bend” symbol in the margin. 


= New marginal notes make connections, add interesting notes, and provide advice to 
students. 


® More details and added explanations, in both proofs and exposition, make it easier for 
students to read the book. 


m™ Many new exercises, both routine and challenging, have been added, while many ex- 
isting exercises have been improved. 


Enhanced Coverage of Logic, Sets, and Proof 


® The satisfiability problem is addressed in greater depth, with Sudoku modeled in terms 
of satisfiability. 


® Hilbert’s Grand Hotel is used to help explain uncountability. 


® Proofs throughout the book have been made more accessible by adding steps and reasons 
behind these steps. 


= A template for proofs by mathematical induction has been added. 


® The step that applies the inductive hypothesis in mathematical induction proof is now 
explicitly noted. 


Algorithms 
™ The pseudocode used in the book has been updated. 


= Explicit coverage of algorithmic paradigms, including brute force, greedy algorithms, 
and dynamic programing, is now provided. 


= Useful rules for big-O estimates of logarithms, powers, and exponential functions have 
been added. 


Number Theory and Cryptography 


m= Expanded coverage allows instructors to include just a little or a lot of number theory 
in their courses. 


® The relationship between the mod function and congruences has been explained more 
fully. 


® The sieve of Eratosthenes is now introduced earlier in the book. 
® Linear congruences and modular inverses are now covered in more detail. 


= Applications of number theory, including check digits and hash functions, are covered 
in great depth. 


= A new section on cryptography integrates previous coverage, and the notion of a cryp- 
tosystem has been introduced. 


® Cryptographic protocols, including digital signatures and key sharing, are now covered. 
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Graph Theory 


@ A structured introduction to graph theory applications has been added. 
= More coverage has been devoted to the notion of social networks. 


= Applications to the biological sciences and motivating applications for graph isomor- 
phism and planarity have been added. 


= Matchings in bipartite graphs are now covered, including Hall’s theorem and its proof. 


™ Coverage of vertex connectivity, edge connectivity, and n-connectedness has been 
added, providing more insight into the connectedness of graphs. 


Enrichment Material 


™ Many biographies have been expanded and updated, and new biographies of Bellman, 
Bézout Bienyamé, Cardano, Catalan, Cocks, Cook, Dirac, Hall, Hilbert, Ore, and Tao 
have been added. 


® Historical information has been added throughout the text. 


= Numerous updates for latest discoveries have been made. 


Expanded Media 


® Extensive effort has been devoted to producing valuable web resources for this book. 
m= Extra examples in key parts of the text have been provided on companion website. 


® Interactive algorithms have been developed, with tools for using them to explore topics 
and for classroom use. 


= A new online ancillary, The Virtual Discrete Mathematics Tutor, available in fall 2012, 
will help students overcome problems learning discrete mathematics. 


= Anew homework delivery system, available in fall 2012, will provide automated home- 
work for both numerical and conceptual exercises. 


™ Student assessment modules are available for key concepts. 
= Powerpoint transparencies for instructor use have been developed. 


= Asupplement Exploring Discrete Mathematics has been developed, providing extensive 
support for using Maple™ or Mathematica™ in conjunction with the book. 


= An extensive collection of external web links is provided. 


Features of the Book 


ACCESSIBILITY This text has proved to be easily read and understood by beginning 
students. There are no mathematical prerequisites beyond college algebra for almost all the 
content of the text. Students needing extra help will find tools on the companion website for 
bringing their mathematical maturity up to the level of the text. The few places in the book 
where calculus is referred to are explicitly noted. Most students should easily understand the 
pseudocode used in the text to express algorithms, regardless of whether they have formally 
studied programming languages. There is no formal computer science prerequisite. 


Each chapter begins at an easily understood and accessible level. Once basic mathematical 


concepts have been carefully developed, more difficult material and applications to other areas 
of study are presented. 
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FLEXIBILITY This text has been carefully designed for flexible use. The dependence 
of chapters on previous material has been minimized. Each chapter is divided into sections of 
approximately the same length, and each section is divided into subsections that form natural 
blocks of material for teaching. Instructors can easily pace their lectures using these blocks. 


WRITING STYLE The writing style in this book is direct and pragmatic. Precise mathe- 
matical language is used without excessive formalism and abstraction. Care has been taken to 
balance the mix of notation and words in mathematical statements. 


MATHEMATICAL RIGOR AND PRECISION All definitions and theorems in this text 
are stated extremely carefully so that students will appreciate the precision of language and 
rigor needed in mathematics. Proofs are motivated and developed slowly; their steps are all 
carefully justified. The axioms used in proofs and the basic properties that follow from them 
are explicitly described in an appendix, giving students a clear idea of what they can assume in 
a proof. Recursive definitions are explained and used extensively. 


WORKED EXAMPLES Over 800 examples are used to illustrate concepts, relate different 
topics, and introduce applications. In most examples, a question is first posed, then its solution 
is presented with the appropriate amount of detail. 


APPLICATIONS The applications included in this text demonstrate the utility of discrete 
mathematics in the solution of real-world problems. This text includes applications to a wide va- 
riety of areas, including computer science, data networking, psychology, chemistry, engineering, 
linguistics, biology, business, and the Internet. 


ALGORITHMS Results in discrete mathematics are often expressed in terms of algo- 
rithms; hence, key algorithms are introduced in each chapter of the book. These algorithms 
are expressed in words and in an easily understood form of structured pseudocode, which is 
described and specified in Appendix 3. The computational complexity of the algorithms in the 
text is also analyzed at an elementary level. 


HISTORICAL INFORMATION The background of many topics is succinctly described 
in the text. Brief biographies of 83 mathematicians and computer scientists are included as foot- 
notes. These biographies include information about the lives, careers, and accomplishments of 
these important contributors to discrete mathematics and images, when available, are displayed. 

In addition, numerous historical footnotes are included that supplement the historical in- 
formation in the main body of the text. Efforts have been made to keep the book up-to-date by 
reflecting the latest discoveries. 


KEY TERMS AND RESULTS A list of key terms and results follows each chapter. The 
key terms include only the most important that students should learn, and not every term defined 
in the chapter. 


EXERCISES There are over 4000 exercises in the text, with many different types of 
questions posed. There is an ample supply of straightforward exercises that develop basic skills, 
a large number of intermediate exercises, and many challenging exercises. Exercises are stated 
clearly and unambiguously, and all are carefully graded for level of difficulty. Exercise sets 
contain special discussions that develop new concepts not covered in the text, enabling students 
to discover new ideas through their own work. 

Exercises that are somewhat more difficult than average are marked with a single star *; 
those that are much more challenging are marked with two stars **. Exercises whose solutions 
require calculus are explicitly noted. Exercises that develop results used in the text are clearly 
identified with the right pointing hand symbol fs. Answers or outlined solutions to all odd- 
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numbered exercises are provided at the back of the text. The solutions include proofs in which 
most of the steps are clearly spelled out. 


REVIEW QUESTIONS A set of review questions is provided at the end of each chapter. 
These questions are designed to help students focus their study on the most important concepts 
and techniques of that chapter. To answer these questions students need to write long answers, 
rather than just perform calculations or give short replies. 


SUPPLEMENTARY EXERCISE SETS Each chapter is followed by a rich and varied 
set of supplementary exercises. These exercises are generally more difficult than those in the 
exercise sets following the sections. The supplementary exercises reinforce the concepts of the 
chapter and integrate different topics more effectively. 


COMPUTER PROJECTS Each chapter is followed by a set of computer projects. The 
approximately 150 computer projects tie together what students may have learned in computing 
and in discrete mathematics. Computer projects that are more difficult than average, from both 
a mathematical and a programming point of view, are marked with a star, and those that are 
extremely challenging are marked with two stars. 


COMPUTATIONS AND EXPLORATIONS A set of computations and explorations is 
included at the conclusion of each chapter. These exercises (approximately 120 in total) are de- 
signed to be completed using existing software tools, such as programs that students or instruc- 
tors have written or mathematical computation packages such as Maple™ or Mathematica™. 
Many of these exercises give students the opportunity to uncover new facts and ideas through 
computation. (Some of these exercises are discussed in the Exploring Discrete Mathematics 


companion workbooks available online.) 


WRITING PROJECTS Each chapter is followed by a set of writing projects. To do these 
projects students need to consult the mathematical literature. Some of these projects are historical 
in nature and may involve looking up original sources. Others are designed to serve as gateways 
to new topics and ideas. All are designed to expose students to ideas not covered in depth in 
the text. These projects tie mathematical concepts together with the writing process and help 
expose students to possible areas for future study. (Suggested references for these projects can 
be found online or in the printed Student’s Solutions Guide.) 


APPENDIXES There are three appendixes to the text. The first introduces axioms for real 
numbers and the positive integers, and illustrates how facts are proved directly from these axioms. 
The second covers exponential and logarithmic functions, reviewing some basic material used 
heavily in the course. The third specifies the pseudocode used to describe algorithms in this text. 


SUGGESTED READINGS A list of suggested readings for the overall book and for each 
chapter is provided after the appendices. These suggested readings include books at or below 
the level of this text, more difficult books, expository articles, and articles in which discoveries 
in discrete mathematics were originally published. Some of these publications are classics, 
published many years ago, while others have been published in the last few years. 


How to Use This Book 


This text has been carefully written and constructed to support discrete mathematics courses 
at several levels and with differing foci. The following table identifies the core and optional 
sections. An introductory one-term course in discrete mathematics at the sophomore level can 
be based on the core sections of the text, with other sections covered at the discretion of the 
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instructor. A two-term introductory course can include all the optional mathematics sections in 
addition to the core sections. A course with a strong computer science emphasis can be taught 
by covering some or all of the optional computer science sections. Instructors can find sample 
syllabi for a wide range of discrete mathematics courses and teaching suggestions for using each 
section of the text can be found in the Instructor’s Resource Guide available on the website for 
this book. 


1.1-1.8 (as needed) 
2.1-2.4, 2.6 (as needed) 
3.1-3.3 (as needed) 


4.1-4.4 (as needed) 


12.1-12.4 
13.1-13.5 


Instructors using this book can adjust the level of difficulty of their course by choosing 
either to cover or to omit the more challenging examples at the end of sections, as well as 
the more challenging exercises. The chapter dependency chart shown here displays the strong 
dependencies. A star indicates that only relevant sections of the chapter are needed for study of a 
later chapter. Weak dependencies have been ignored. More details can be found in the Instructor 
Resource Guide. 


Chapter 1 
ieee Oe 
Chapter 12 
Chapter 3* 
Chapter 9* 
| Chapter 4* 
Chapter 10* Chapter 13 
| Chapter 5* 
Chapter 11 
Chapter 6* 
Chapter 7 Chapter 8 


STUDENT’S SOLUTIONS GUIDE This student manual, available separately, contains 
full solutions to all odd-numbered problems in the exercise sets. These solutions explain why 
a particular method is used and why it works. For some exercises, one or two other possible 
approaches are described to show that a problem can be solved in several different ways. Sug- 
gested references for the writing projects found at the end of each chapter are also included in 
this volume. Also included are a guide to writing proofs and an extensive description of common 
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mistakes students make in discrete mathematics, plus sample tests and a sample crib sheet for 
each chapter designed to help students prepare for exams. 

(ISBN-10: 0-07-735350-1) (ISBN-13: 978-0-07-735350-6) 

INSTRUCTOR’S RESOURCE GUIDE This manual, available on the website and in 
printed form by request for instructors, contains full solutions to even-numbered exercises in 
the text. Suggestions on how to teach the material in each chapter of the book are provided, 
including the points to stress in each section and how to put the material into perspective. It 
also offers sample tests for each chapter and a test bank containing over 1500 exam questions to 
choose from. Answers to all sample tests and test bank questions are included. Finally, several 
sample syllabi are presented for courses with differing emphases and student ability levels. 


(ISBN-10: 0-07-735349-8) (ISBN-13: 978-0-07-735349-0) 
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The Companion Website 


T he extensive companion website accompanying this text has been substantially enhanced 
for the seventh edition This website is accessible at www.mlhhe.com/rosen. The homepage 
shows the Information Center, and contains login links for the site’s Student Site and Instructor 
Site. Key features of each area are described below: 


THE INFORMATION CENTER 


The Information Center contains basic information about the book including the expanded 
table of contents (including subsection heads), the preface, descriptions of the ancillaries, and 
a sample chapter. It also provides a link that can be used to submit errata reports and other 
feedback about the book. 


STUDENT SITE 


The Student site contains a wealth of resources available for student use, including the 
following, tied into the text wherever the special icons displayed below are found in the text: 


m= Extra Examples Youcan find a large number of additional examples on the site, covering 
all chapters of the book. These examples are concentrated in areas where students often 
ask for additional material. Although most of these examples amplify the basic concepts, 
more-challenging examples can also be found here. 


m Interactive Demonstration Applets These applets enable you to interactively explore 
how important algorithms work, and are tied directly to material in the text with linkages to 
examples and exercises. Additional resources are provided on how to use and apply these 
applets. 


m Self Assessments These interactive guides help you assess your understanding of 14 key 
concepts, providing a question bank where each question includes a brief tutorial followed 
by a multiple-choice question. If you select an incorrect answer, advice is provided to help 
you understand your error. Using these Self Assessments, you should be able to diagnose 
your problems and find appropriate help. 


m Web Resources Guide This guide provides annotated links to hundreds of external websites 
containing relevant material such as historical and biographical information, puzzles and 
problems, discussions, applets, programs, and more. These links are keyed to the text by page 
number. 


Additional resources in the Student site include: 


= Exploring Discrete Mathematics This ancillary provides help for using a computer alge- 
bra system to do a wide range of computations in discrete mathematics. Each chapter provides 
a description of relevant functions in the computer algebra system and how they are used, pro- 
grams to carry out computations in discrete mathematics, examples, and exercises that can be 
worked using this computer algebra system. Two versions, Exploring Discrete Mathematics 
with Maple™ and Exploring Discrete Mathematics with Mathematica™ will be available. 


» Applications of Discrete Mathematics — This ancillary contains 24 chapters—each with 
its own set of exercises—presenting a wide variety of interesting and important applications 
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covering three general areas in discrete mathematics: discrete structures, combinatorics, and 
graph theory. These applications are ideal for supplementing the text or for independent study. 


« A Guide to Proof-Writing — This guide provides additional help for writing proofs, a skill 
that many students find difficult to master. By reading this guide at the beginning of the 
course and periodically thereafter when proof writing is required, you will be rewarded as 
your proof-writing ability grows. (Also available in the Student’s Solutions Guide.) 


= Common Mistakes in Discrete Mathematics This guide includes a detailed list of com- 
mon misconceptions that students of discrete mathematics often have and the kinds of errors 
they tend to make. You are encouraged to review this list from time to time to help avoid these 
common traps. (Also available in the Student’s Solutions Guide.) 


» Advice on Writing Projects This guide offers helpful hints and suggestions for the Writing 
Projects in the text, including an extensive bibliography of helpful books and articles for 
research; discussion of various resources available in print and online; tips on doing library 
research; and suggestions on how to write well. (Also available in the Student’s Solutions 
Guide.) 


« The Virtual Discrete Mathematics Tutor This extensive ancillary provides students with 
valuable assistance as they make the transition from lower-level courses to discrete mathemat- 
ics. The errors students have made when studying discrete mathematics using this text has been 
analyzed to design this resource. Students will be able to get many of their questions answered 
and can overcome many obstacles via this ancillaries. The Virtual Discrete Mathematics Tutor 
is expected to be available in the fall of 2012. 


INSTRUCTOR SITE 


This part of the website provides access to all of the resources on the Student Site, as well as 
these resources for instructors: 


» Suggested Syllabi Detailed course outlines are shown, offering suggestions for courses 
with different emphases and different student backgrounds and ability levels. 


= Teaching Suggestions This guide contains detailed teaching suggestions for instructors, 
including chapter overviews for the entire text, detailed remarks on each section, and comments 
on the exercise sets. 


= Printable Tests Printable tests are offered in TeX and Word format for every chapter, and 
can be customized by instructors. 


« PowerPoints Lecture Slides and PowerPoint Figures and Tables Anextensive collection 
of PowerPoint slides for all chapters of the text are provided for instructor use. In addition, 
images of all figures and tables from the text are provided as PowerPoint slides. 


« Homework Delivery System Anextensive homework delivery system, under development 
for availability in fall 2012, will provide questions tied directly to the text, so that students 
will be able to do assignments on-line. Moreover, they will be able to use this system in a 
tutorial mode. This system will be able to automatically grade assignments, and deliver free- 
form student input to instructors for their own analysis. Course management capabilities will 
be provided that will allow instructors to create assignments, automatically assign and grade 
homework, quiz, and test questions from a bank of questions tied directly to the text, create 
and edit their own questions, manage course announcements and due dates, and track student 
progress. 
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To the Student 


W hat is discrete mathematics ? Discrete mathematics is the part of mathematics devoted to 
the study of discrete objects. (Here discrete means consisting of distinct or unconnected 
elements.) The kinds of problems solved using discrete mathematics include: 


= How many ways are there to choose a valid password on a computer system? 
& What is the probability of winning a lottery? 

@ Is there a link between two computers in a network? 

= How can I identify spam e-mail messages? 

= How can I encrypt a message so that no unintended recipient can read it? 

m& What is the shortest path between two cities using a transportation system? 

= How can a list of integers be sorted so that the integers are in increasing order? 
= How many steps are required to do such a sorting? 

= How can it be proved that a sorting algorithm correctly sorts a list? 

= How can a circuit that adds two integers be designed? 

= How many valid Internet addresses are there? 


You will learn the discrete structures and techniques needed to solve problems such as these. 

More generally, discrete mathematics is used whenever objects are counted, when relation- 
ships between finite (or countable) sets are studied, and when processes involving a finite number 
of steps are analyzed. A key reason for the growth in the importance of discrete mathematics is 
that information is stored and manipulated by computing machines in a discrete fashion. 


WHY STUDY DISCRETE MATHEMATICS? There are several important reasons for 
studying discrete mathematics. First, through this course you can develop your mathematical 
maturity: that is, your ability to understand and create mathematical arguments. You will not get 
very far in your studies in the mathematical sciences without these skills. 

Second, discrete mathematics is the gateway to more advanced courses in all parts of 
the mathematical sciences. Discrete mathematics provides the mathematical foundations for 
many computer science courses including data structures, algorithms, database theory, automata 
theory, formal languages, compiler theory, computer security, and operating systems. Students 
find these courses much more difficult when they have not had the appropriate mathematical 
foundations from discrete math. One student has sent me an e-mail message saying that she 
used the contents of this book in every computer science course she took! 

Math courses based on the material studied in discrete mathematics include logic, set theory, 
number theory, linear algebra, abstract algebra, combinatorics, graph theory, and probability 
theory (the discrete part of the subject). 

Also, discrete mathematics contains the necessary mathematical background for solving 
problems in operations research (including many discrete optimization techniques), chemistry, 
engineering, biology, and so on. In the text, we will study applications to some of these areas. 

Many students find their introductory discrete mathematics course to be significantly more 
challenging than courses they have previously taken. One reason for this is that one of the 
primary goals of this course is to teach mathematical reasoning and problem solving, rather 
than a discrete set of skills. The exercises in this book are designed to reflect this goal. Although 
there are plenty of exercises in this text similar to those addressed in the examples, a large 
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percentage of the exercises require original thought. This is intentional. The material discussed 
in the text provides the tools needed to solve these exercises, but your job is to successfully 
apply these tools using your own creativity. One of the primary goals of this course is to learn 
how to attack problems that may be somewhat different from any you may have previously 
seen. Unfortunately, learning how to solve only particular types of exercises is not sufficient for 
success in developing the problem-solving skills needed in subsequent courses and professional 
work. This text addresses many different topics, but discrete mathematics is an extremely diverse 
and large area of study. One of my goals as an author is to help you develop the skills needed 
to master the additional material you will need in your own future pursuits. 


THE EXERCISES I would like to offer some advice about how you can best learn discrete 
mathematics (and other subjects in the mathematical and computing sciences). You will learn the 
most by actively working exercises. I suggest that you solve as many as you possibly can. After 
working the exercises your instructor has assigned, I encourage you to solve additional exercises 
such as those in the exercise sets following each section of the text and in the supplementary 
exercises at the end of each chapter. (Note the key explaining the markings preceding exercises.) 


Key to the Exercises 


no marking A routine exercise 


* A difficult exercise 


si An extremely challenging exercise 


= An exercise containing a result used in the book (Table 1 on the 
following page shows where these exercises are used.) 


(Requires calculus) An exercise whose solution requires the use of limits or concepts 
from differential or integral calculus 


The best approach is to try exercises yourself before you consult the answer section at the 
end of this book. Note that the odd-numbered exercise answers provided in the text are answers 
only and not full solutions; in particular, the reasoning required to obtain answers is omitted in 
these answers. The Student’s Solutions Guide, available separately, provides complete, worked 
solutions to all odd-numbered exercises in this text. When you hit an impasse trying to solve an 
odd-numbered exercise, I suggest you consult the Student’s Solutions Guide and look for some 
guidance as to how to solve the problem. The more work you do yourself rather than passively 
reading or copying solutions, the more you will learn. The answers and solutions to the even- 
numbered exercises are intentionally not available from the publisher; ask your instructor if you 
have trouble with these. 


WEB RESOURCES You are strongly encouraged to take advantage of additional re- 
sources available on the Web, especially those on the companion website for this book found 
at www.mhhe.com/osen. You will find many Extra Examples designed to clarify key concepts; 
Self Assessments for gauging how well you understand core topics; Interactive Demonstration 
Applets exploring key algorithms and other concepts; a Web Resources Guide containing an 
extensive selection of links to external sites relevant to the world of discrete mathematics; extra 
explanations and practice to help you master core concepts; added instruction on writing proofs 
and on avoiding common mistakes in discrete mathematics; in-depth discussions of important 
applications; and guidance on utilizing Maple™ software to explore the computational aspects 
of discrete mathematics. Places in the text where these additional online resources are available 
are identified in the margins by special icons. You will also find (after fall 2012) the Virtual 
Discrete Mathematics Tutor, an on-line resource that provides extra support to help you make 
the transition from lower level courses to discrete mathematics. This tutorial should help answer 
many of your questions and correct errors that you may make, based on errors other students 
using this book, have made. For more details on these and other online resources, see the 
description of the companion website immediately preceding this “To the Student” message. 
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THE VALUE OF THIS BOOK My intention is to make your substantial investment in 
this text an excellent value. The book, the associated ancillaries, and companion website have 
taken many years of effort to develop and refine. I am confident that most of you will find that 
the text and associated materials will help you master discrete mathematics, just as so many 
previous students have. Even though it is likely that you will not cover some chapters in your 
current course, you should find it helpful—as many other students have—to read the relevant 
sections of the book as you take additional courses. Most of you will return to this book as a 
useful tool throughout your future studies, especially for those of you who continue in computer 
science, mathematics, and engineering. I have designed this book to be a gateway for future 
studies and explorations, and to be comprehensive reference, and I wish you luck as you begin 
your journey. 


Kenneth H. Rosen 
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e Foundations: 
Logic and Proofs 


T he rules of logic specify the meaning of mathematical statements. For instance, these rules 
help us understand and reason with statements such as “There exists an integer that is 
not the sum of two squares” and “For every positive integer n, the sum of the positive integers 
not exceeding n is n(n + 1)/2.” Logic is the basis of all mathematical reasoning, and of all 
automated reasoning. It has practical applications to the design of computing machines, to the 
specification of systems, to artificial intelligence, to computer programming, to programming 
languages, and to other areas of computer science, as well as to many other fields of study. 


To understand mathematics, we must understand what makes up @/€orrect mathematical 


collection of theorems on a topic organize what we know about this topic. To learn a mathematical 
topic, a person needs to actively construct mathematical arguments on this topic, and not just 
read exposition. Moreover, knowing the proof of a theorem often makes it possible to modify 
the result to fit new situations. 


Everyone knows that proofs are important throughout mathematics, but many people find 
it surprising how In fact, proofs are used to verify 
that computer programs produce the correct output for all possible input values, to show that 


(artificial intelligence) Furthermore, automated reasoning systems have been created to allow 
computers to construct their own proofs. 


In this chapter, we will explain what makes up a correct mathematical argument and intro- 
duce tools to construct these arguments. We will develop an arsenal of different proof methods 
that will enable us to prove many different types of results. After introducing many different 
methods of proof, we will introduce several strategies for constructing proofs. We will intro- 
duce the notion of a conjecture and explain the process of developing mathematics by studying 
conjectures. 


Propositional Logic 


Introduction 


The rules of logic give precise meaning to mathematical statements. These rules are used to 

Because a major goal of this book 
is to teach the reader how to understand and how to construct correct mathematical arguments, 
we begin our study of discrete mathematics with an introduction to logic. 


Besides the importance of logic in understanding mathematical reasoning, logic has numer 


hese rules are used in the 
the and in 


many other ways. Furthermore, software systems have been developed for constructing some, 
but not all, types of proofs automatically. We will discuss these applications of logic in this and 
later chapters. 
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1 / The Foundations: Logic and Proofs 


Propositions 


Our discussion begins with an introduction to the basic building blocks of logic—propositions. 
A proposition is a declarative sentence (that is, a sentence that declares a fact) that is either true 
or false, but not both. 


EXAMPLE1_ All the following declarative sentences are propositions. 


Extra oo 1. Washington, D.C., is the capital of the United States of America. 
Examples @id 2. Toronto is the capital of Canada. 
3.141=2. 
4,.24+2=3. 
Propositions | and 3 are true, whereas 2 and 4 are false. < 


Some sentences that are not propositions are given in Example 2. 


EXAMPLE 2 Consider the following sentences. 


. What time is it? 

. Read this carefully. 
x+1=2. 
x+y=Z. 


Ree 


Sentences | and 2 are not propositions because they are not declarative sentences. Sentences 3 
and 4 are not propositions because they are neither true nor false. Note that each of sentences 3 
and 4 can be turned into a proposition if we assign values to the variables. We will also discuss 
other ways to turn sentences such as these into propositions in Section 1.4. < 


We use letters to denote propositional variables (or statement variables), that is, vari- 
ables that represent propositions, just as letters are used to denote numerical variables. The 


ARISTOTLE (384 B.c.£.-322 B.c.E.) Aristotle was born in Stagirus (Stagira) in northern Greece. His father was 
the personal physician of the King of Macedonia. Because his father died when Aristotle was young, Aristotle 
could not follow the custom of following his father’s profession. Aristotle became an orphan at a young age 
when his mother also died. His guardian who raised him taught him poetry, rhetoric, and Greek. At the age of 
17, his guardian sent him to Athens to further his education. Aristotle joined Plato’s Academy, where for 20 
years he attended Plato’s lectures, later presenting his own lectures on rhetoric. When Plato died in 347 B.cz, 
| Aristotle was not chosen to succeed him because his views differed too much from those of Plato. Instead, 
| Aristotle joined the court of King Hermeas where he remained for three years, and married the niece of the 
" King. When the Persians defeated Hermeas, Aristotle moved to Mytilene and, at the invitation of King Philip 
of Macedonia, he tutored Alexander, Philip’s son, who later became Alexander the Great. Aristotle tutored Alexander for five years 
and after the death of King Philip, he returned to Athens and set up his own school, called the Lyceum. 

Aristotle’s followers were called the peripatetics, which means “to walk about,” because Aristotle often walked around as he 
discussed philosophical questions. Aristotle taught at the Lyceum for 13 years where he lectured to his advanced students in the 
morning and gave popular lectures to a broad audience in the evening. When Alexander the Great died in 323 B.c.E., a backlash against 
anything related to Alexander led to trumped-up charges of impiety against Aristotle. Aristotle fled to Chalcis to avoid prosecution. 
He only lived one year in Chalcis, dying of a stomach ailment in 322 B.c.E. 

Aristotle wrote three types of works: those written for a popular audience, compilations of scientific facts, and systematic 
treatises. The systematic treatises included works on logic, philosophy, psychology, physics, and natural history. Aristotle’s writings 
were preserved by a student and were hidden in a vault where a wealthy book collector discovered them about 200 years later. They 
were taken to Rome, where they were studied by scholars and issued in new editions, preserving them for posterity. 
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conventional letters used for propositional variables are p,q,r,s,.... The truth value of a 
proposition is true, denoted by T, if it is a true proposition, and the truth value of a proposition 
is false, denoted by F, if it is a false proposition. 


The area of oe that deals with Fey geen is called ee ee 


We now turn our attention to methods for producing new propositions from those that 
links id we already have. These methods were discussed by the English mathematician 
any mathematical statements are constructed by 
combining one or more propositions. New propositions, called compound propositions, are 
formed from existing propositions using logical operators. 


DEFINITION 1 Let p bea proposition. The negation of p, denoted by —p (also denoted by 7p), is the statement 


“Tt is not the case that p.” 


The proposition — p is read “not p.” The truth value of the negation of p, —p, is the opposite 
of the truth value of p. 


EXAMPLE 3 _ Find the negation of the proposition 


“Michael’s PC runs Linux” 


Extra p> 
Examples 2 


and express this in simple English. 
Solution: The negation is 
“Tt is not the case that Michael’s PC runs Linux.” 


This negation can be more simply expressed as 


“Michael’s PC does not run Linux.” 


EXAMPLE 4 Find the negation of the proposition 
“Vandana’s smartphone has at least 32GB of memory” 


and express this in simple English. 


Solution: The negation is 

“Tt is not the case that Vandana’s smartphone has at least 32GB of memory.” 
This negation can also be expressed as 

“Vandana’s smartphone does not have at least 32GB of memory” 


or even more simply as 


“Vandana’s smartphone has less than 32GB of memory.” 
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TABLE 1 The 
Truth Table for 
the Negation of a 
Proposition. 


Table 1 displays thé! truth fable) for the negation of a proposition p. This table has a row 
for each of the two possible truth values of a proposition p. Each row shows the truth value of 
—p corresponding to the truth value of p for this row. 

The negation of a proposition can also be considered the result of the operation of the 
negation operator on a proposition. The negation operator constructs a new proposition from 
a single existing proposition. We will now introduce the logical operators that are used to form 
new propositions from two or more existing propositions. These logical operators are also called 
connectives. 


DEFINITION 2. Let p and q be propositions. The conjunction of p and q, denoted by p A q, is the proposition 


“p and q.” The conjunction p A gq is true when both p and gq are true and is false otherwise. 


Table 2 displays the truth table of p A qg. This table has a row for each of the four possible 
combinations of truth values of p and qg. The four rows correspond to the pairs of truth values 
TT, TE, FT, and FF, where the first truth value in the pair is the truth value of p and the second 
truth value is the truth value of q. 

Note that in logic the word “but” sometimes is used instead of “and” in a conjunction. For 
example, the statement “The sun is shining, but it is raining” is another way of saying “The sun 
is shining and it is raining.” (In natural language, there is a subtle difference in meaning between 
“and” and “but”; we will not be concerned with this nuance here.) 


EXAMPLE 5 _ Find the conjunction of the propositions p and g where p is the proposition “Rebecca’s PC has 


more than 16 GB free hard disk space” and q is the proposition “The processor in Rebecca’s 
PC runs faster than 1 GHz.” 


Solution: The conjunction of these propositions, p A q, is the proposition ““Rebecca’s PC has 
more than 16 GB free hard disk space, and the processor in Rebecca’s PC runs faster than | 
GHz.” This conjunction can be expressed more simply as ““Rebecca’s PC has more than 16 GB 
free hard disk space, and its processor runs faster than 1 GHz.” For this conjunction to be true, 
both conditions given must be true. It is false, when one or both of these conditions are false. 


DEFINITION 3. Let p and q be propositions. The disjunction of p and q, denoted by p V q, is the proposition 


“p or q.” The disjunction p V q is false when both p and q are false and is true otherwise. 


Table 3 displays the truth table for p Vv q. 


TABLE 2 The Truth Table for TABLE 3 The Truth Table for 
the Conjunction of Two the Disjunction of Two 
Propositions. Propositions. 
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or is used in English, namely, as a A disjunction is true when at least one of the 
two propositions is true. For instance, the inclusive or is being used in the statement 


The use of the connective or in a inte corresponds to one of the two ways the word 


“Students who have taken calculusfor komputer science can take this class.” 


Here, we mean that students who have taken both calculus and computer science can take the 
class, as well as the students who have taken only one of the two subjects. On the other hand, 
we are using the exclusive or when we say 


“Students who have taken calculus or computer science, but not both, can enroll in this 
class.” 


Here, we mean that students who have taken both calculus and a computer science course cannot 
take the class. Only those who have taken exactly one of the two courses can take the class. 

Similarly, when a menu at a restaurant states, “Soup or salad comes with an entrée,” the 
restaurant almost always means that customers can have either soup or salad, but not both. 
Hence, this is an exclusive, rather than an inclusive, or. 


EXAMPLE 6 What is the disjunction of the propositions p and q where p and q are the same propositions as 
in Example 5? 


Extra * Solution: The disjunction of p and q, p V q, is the proposition 
Examples 


“Rebecca’s PC has at least 16 GB free hard disk space, or the processor in Rebecca’s PC 
runs faster than | GHz.” 


This proposition is true when Rebecca’s PC has at least 16 GB free hard disk space, when the 
PC’s processor runs faster than | GHz, and when both conditions are true. It is false when both 
of these conditions are false, that is, when Rebecca’s PC has less than 16 GB free hard disk 
space and the processor in her PC runs at 1 GHz or slower. < 


As was previously remarked, the use of the connective or in a disjunction corresponds 
to one of the two ways the word or is used in English, namely, in an inclusive way. Thus, a 
disjunction is true when at least one of the two propositions in it is true. Sometimes, we use or 
in an exclusive sense. When the exclusive or is used to connect the propositions p and q, the 
proposition “p or qg (but not both)” is obtained. This proposition is true when p is true and g is 
false, and when p is false and gq is true. It is false when both p and q are false and when both 
are true. 


GEORGE BOOLE (1815-1864) George Boole, the son of a cobbler, was born in Lincoln, England, in 
November 1815. Because of his family’s difficult financial situation, Boole struggled to educate himself while 
supporting his family. Nevertheless, he became one of the most important mathematicians of the 1800s. Although 
he considered a career as a clergyman, he decided instead to go into teaching, and soon afterward opened a 
school of his own. In his preparation for teaching mathematics, Boole—unsatisfied with textbooks of his day— 
decided to read the works of the great mathematicians. While reading papers of the great French mathematician 
Lagrange, Boole made discoveries in the calculus of variations, the branch of analysis dealing with finding 
curves and surfaces by optimizing certain parameters. 

In 1848 Boole published The Mathematical Analysis of Logic, the first of his contributions to symbolic logic. 
In 1849 he was appointed professor of mathematics at Queen’s College in Cork, Ireland. In 1854 he published The Laws of Thought, 
his most famous work. In this book, Boole introduced what is now called Boolean algebra in his honor. Boole wrote textbooks 
on differential equations and on difference equations that were used in Great Britain until the end of the nineteenth century. Boole 
married in 1855; his wife was the niece of the professor of Greek at Queen’s College. In 1864 Boole died from pneumonia, which 
he contracted as a result of keeping a lecture engagement even though he was soaking wet from a rainstorm. 
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TABLE 4 The Truth Table for TABLE 5 The Truth Table for 
the Exclusive Or of Two the Conditional Statement 
Propositions. pq. 


DEFINITION 4 Let p and q be propositions. The exclusive or of p and q, denoted by p © q, is the proposition 
that is true when exactly one of p and q is true and is false otherwise. 


The truth table for the exclusive or of two propositions is displayed in Table 4. 


Conditional Statements 


We will discuss several other important ways in which propositions can be combined. 


DEFINITION 5 Let p and q be propositions. The conditional statement p — q is the proposition “if p, then 
q.’ The conditional statement p — q is false when p is true and q is false, and true otherwise. 
In the conditional statement p — q, p is called the hypothesis (or antecedent or premise) 
and q is called the conclusion (or consequence). 


The statement p — g is called a conditional statement because p — g asserts that q is true 
Assessment Q& on the condition that p holds. A conditional statement is also called an implication. 

The truth table for the conditional statement p — q is shown in Table 5. Note that the 
statement p — q is true when both p and gq are true and when p is false (no matter what truth 
value g has). 

Because conditional statements play such an essential role in mathematical reasoning, a 
variety of terminology is used to express p — gq. You will encounter most if not all of the 
following ways to express this conditional statement: 


“if p, then g” “p implies q” 

“if p,q” “p only if q” 

“p is sufficient for q” “a sufficient condition for g is p” 
“q if p” “q whenever p” 

“q when p” “q is necessary for p” 

“a necessary condition for p is q” “q follows from p” 


“q unless =p” 


A useful way to understand the truth value of a conditional statement is to think of an 
obligation or a contract. For example, the pledge many politicians make when running for office 
is 


“Tf I am elected, then I will lower taxes.” 


You might have trouble 
understanding how 
“unless” is used in 
conditional statements 
unless you read this 
paragraph carefully. 


EXAMPLE 7 


Extra pe 
Examples 2 
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If the politician is elected, voters would expect this politician to lower taxes. Furthermore, if the 
politician is not elected, then voters will not have any expectation that this person will lower 
taxes, although the person may have sufficient influence to cause those in power to lower taxes. 
It is only when the politician is elected but does not lower taxes that voters can say that the 
politician has broken the campaign pledge. This last scenario corresponds to the case when p 
is true but g is false in p > q. 

Similarly, consider a statement that a professor might make: 


“If you get 100% on the final, then you will get an A.” 


If you manage to get a 100% on the final, then you would expect to receive an A. If you do not 
get 100% you may or may not receive an A depending on other factors. However, if you do get 
100%, but the professor does not give you an A, you will feel cheated. 

Of the various ways to express the conditional statement p — q, the two that seem to cause 
the most confusion are “p only if g” and “g unless sp.” Consequently, we will provide some 
guidance for clearing up this confusion. 

To remember that “p only if g” expresses the same thing as “if p, then g,” note that “p only 
if q” says that p cannot be true when gq is not true. That is, the statement is false if p is true, 
but q is false. When p is false, g may be either true or false, because the statement says nothing 
about the truth value of g. Be careful not to use “g only if p” to express p — q because this is 
incorrect. To see this, note that the true values of “g only if p” and p — gq are different when 
p and q have different truth values. 

To remember that “gq unless —p” expresses the same conditional statement as “if p, then 
q;’ note that “gq unless =p” means that if —p: is false, then g must be true. That is, the statement 
“q unless sp” is false when p is true but g is false, but it is true otherwise. Consequently, 
“q unless =p” and p — q always have the same truth value. 

We illustrate the translation between conditional statements and English statements in Ex- 
ample 7. 


Let p be the statement ‘“‘Maria learns discrete mathematics” and g the statement “Maria will 


find a good job.” Express the statement p — q as a statement in English. 


Solution: From the definition of conditional statements, we see that when p is the statement 
“Maria learns discrete mathematics” and q is the statement “Maria will find a good job,” p > q 
represents the statement 


“Tf Maria learns discrete mathematics, then she will find a good job.” 


There are many other ways to express this conditional statement in English. Among the most 
natural of these are: 


“Maria will find a good job when she learns discrete mathematics.” 
“For Maria to get a good job, it is sufficient for her to learn discrete mathematics.” 
and 


“Maria will find a good job unless she does not learn discrete mathematics.” 4 


Note that the way we have defined conditional statements is more general than the meaning 
attached to such statements in the English language. For instance, the conditional statement in 
Example 7 and the statement 


“Tf it is sunny, then we will go to the beach.” 


are statements used in normal language where there is a relationship between the hypothesis 
and the conclusion. Further, the first of these statements is true unless Maria learns discrete 
mathematics, but she does not get a good job, and the second is true unless it is indeed sunny, 
but we do not go to the beach. On the other hand, the statement 
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EXAMPLE 8 


Remember that the 
contrapositive, but neither 
the converse or inverse, of 
a conditional statement is 
equivalent to it. 


“Tf Juan has a smartphone, then 2 + 3 = 5” 


is true from the definition of a conditional statement, because its conclusion is true. (The truth 
value of the hypothesis does not matter then.) The conditional statement 


“Tf Juan has a smartphone, then 2 + 3 = 6” 


is true if Juan does not have a smartphone, even though 2 + 3 = 6 is false. We would not use 
these last two conditional statements in natural language (except perhaps in sarcasm), because 
there is no relationship between the hypothesis and the conclusion in either statement. In math- 
ematical reasoning, we consider conditional statements of a more general sort than we use in 
English. The mathematical concept of a conditional statement is independent of a cause-and- 
effect relationship between hypothesis and conclusion. Our definition of a conditional statement 
specifies its truth values; it is not based on English usage. Propositional language is an artificial 
language; we only parallel English usage to make it easy to use and remember. 

The if-then construction used in many programming languages is different from that used 
in logic. Most programming languages contain statements such as if p then S, where p is a 
proposition and S is a program segment (one or more statements to be executed). When execution 
of a program encounters such a statement, S is executed if p is true, but S is not executed if p 
is false, as illustrated in Example 8. 


What is the value of the variable x after the statement 


if2+2=4thenx:=x+1 


if x = 0 before this statement is encountered? (The symbol := stands for assignment. The 
statement x := x + | means the assignment of the value of x + 1 to x.) 


Solution: Because 2 + 2 = 4 is true, the assignment statement x := x + | is executed. Hence, 
x has the value 0 + 1 = 1 after this statement is encountered. < 


CONVERSE, CONTRAPOSITIVE, AND INVERSE We can form some new conditional 
statements starting with a conditional statement p — q. In particular, there are three related 
conditional statements that occur so often that they have special names. The propo 


conditional statements formed from p — gq, only the contrapositive always has the same truth 
value as p > q. 

We first show that the contrapositive, ~g — —p, of a conditional statement p — g always 
has the same truth value as p — q. To see this, note that the contrapositive is false only when 
—p is false and —g is true, that is, only when p is true and q is false. We now show that neither 
the converse, g — p, nor the inverse, —p — -q, has the same truth value as p > q for all 
possible truth values of p and q. Note that when p is true and gq is false, the original conditional 
statement is false, but the converse and the inverse are both true. 

When two compound propositions always have the same truth value we call them equiv- 
alent, so that a conditional statement and its contrapositive are equivalent. The @dnverse and 

as the reader can verify, but neither is 
equivalent to the original conditional statement. (We will study equivalent propositions in Sec- 
tion 1.3.) Take note that one of the most common logical errors is to assume that the converse 
or the inverse of a conditional statement is equivalent to this conditional statement. 

We illustrate the use of conditional statements in Example 9. 


1.1 Propositional Logic 9 


EXAMPLE 9 What are the contrapositive, the converse, and the inverse of the conditional statement 


Extra p> 
Examples 


“The home team wins whenever it is raining?” 


Solution: Because “gq whenever p” is one of the ways to express the conditional statement 
Dp — q, the original statement can be rewritten as 


“Tf it is raining, then the home team wins.” 
Consequently, the @ontrapositive of this conditional statement is 


“Tf the home team does not win, then it is not raining.” 


The (GORVEFS® is 


“Tf the home team wins, then it is raining.” 


The VERSO) is 
“Tf it is not raining, then the home team does not win.” 


Only the contrapositive is equivalent to the original statement. < 


BICONDITIONALS | We now introduce another way to combine propositions that expresses 
that two propositions have the same truth value. 


Let p and q be propositions. The biconditional statement p <> q is the proposition “‘p if 
and only if g.” The biconditional statement p < gq is true when p and q have the same truth 
values, and is false otherwise. Biconditional statements are also called bi-implications. 


The truth table for p < q is shown in Table 6. Note that the statement p < q is true when both 
the conditional statements p — gq andg — pare true and is false otherwise. That is why we use 
the words “if and only if” to express this logical connective and why it is symbolically written 
by combining the symbols — and <. There are some other common ways to express p <> q: 


“p is necessary and sufficient for g” 
“if p then q, and conversely” 
“p iff q? 


The last way of expressing the biconditional statement p < q uses the abbreviation “iff” for 
“if and only if.” Note that p < q has exactly the same truth value as (p > g) A (q —> Pp). 


TABLE 6 The Truth Table for the 
Biconditional p  q. 
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EXAMPLE 10 


Extra & 
Examples Qin 


EXAMPLE 11 


Let p be the statement “You can take the flight,” and let g be the statement “You buy a ticket.” 
Then p < q is the statement 


“You can take the flight if and only if you buy a ticket.” 


This statement is true if p and q are either both true or both false, that is, if you buy a ticket and 
can take the flight or if you do not buy a ticket and you cannot take the flight. It is false when 
p and q have opposite truth values, that is, when you do not buy a ticket, but you can take the 
flight (such as when you get a free trip) and when you buy a ticket but you cannot take the flight 
(such as when the airline bumps you). < 


IMPLICIT USE OF BICONDITIONALS You should be aware that biconditionals are not 
always explicit in natural language. In particular, the “if and only if” construction used in 
biconditionals is rarely used in common language. Instead, biconditionals are often expressed 
using an “if, then” or an “only if” construction. The other part of the “if and only if” is implicit. 
That is, the converse is implied, but not stated. For example, consider the statement in English 
“If you finish your meal, then you can have dessert.” What is really meant is “You can have 
dessert if and only if you finish your meal.” This last statement is logically equivalent to the 
two statements “If you finish your meal, then you can have dessert” and “You can have dessert 
only if you finish your meal.’ Because of this imprecision in natural language, we need to 
make an assumption whether a conditional statement in natural language implicitly includes its 
converse. Because precision is essential in mathematics and in logic, we will always distinguish 
between the conditional statement p — g and the biconditional statement p <= q. 


Truth Tables of Compound Propositions 


We have now introduced four important logical connectives—conjunctions, disjunctions, con- 
ditional statements, and biconditional statements—as well as negations. We can use these con- 
nectives to build up complicated compound propositions involving any number of propositional 
variables. We can use truth tables to determine the truth values of these compound propositions, 
as Example 11 illustrates. We use a separate column to find the truth value of each compound 
expression that occurs in the compound proposition as it is built up. The truth values of the 
compound proposition for each combination of truth values of the propositional variables in it 
is found in the final column of the table. 


Construct the truth table of the compound proposition 


(pV 7q) > (Pq). 


Solution: Because this truth table involves two propositional variables p and q, there are four 
rows in this truth table, one for each of the pairs of truth values TT, TF, FT, and FF. The first 
two columns are used for the truth values of p and q, respectively. In the third column we find 
the truth value of —q, needed to find the truth value of p V —q, found in the fourth column. The 
fifth column gives the truth value of p A q. Finally, the truth value of (p V ~qg) > (p Aq) 1s 
found in the last column. The resulting truth table is shown in Table 7. < 


TABLE 7 The Truth Table of (p V —q) > (pq). 
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Precedence of Logical Operators 


We can construct compound propositions using the negation operator and the logical operators 
defined so far. We will generally use parentheses to specify the order in which logical operators 
in a compound proposition are to be applied. For instance, (p V gq) A (=r) is the conjunction 
of p Vv q and =r. However, to reduce the number of parentheses, we specify that the negation 
Precedence | operator is applied before all other logical operators. This means that =p A q is the conjunction 
of =p andq,namely, (=p) A q, not the negation of the conjunction of p andqg, namely —=(p A q). 

Another general rule of precedence is that the conjunction operator takes precedence over 


A 2 the disjunction operator, so that p A qg V r means (p A q) V r rather than p A (g Vr). Because 
Vv 3 this rule may be difficult to remember, we will continue to use parentheses so that the order of 
=> 4 
o 2 


TABLE 8 


Precedence of 
Logical Operators. 


the disjunction and conjunction operators is clear. 

Finally, it is an accepted rule that the conditional and biconditional operators — and <= 
have lower precedence than the conjunction and disjunction operators, A and V. Consequently, 
DV q-—r is the same as (p V g) — r. We will use parentheses when the order of the con- 
ditional operator and biconditional operator is at issue, although the conditional operator has 
precedence over the biconditional operator. Table 8 displays the precedence levels of the logical 
operators, =, A, V, >, and<. 


Logic and Bit Operations 


Computers represent information using bits. ABibis a symbol with two possible values, namely, 

0 (zero) and 1 (one). This meaning of the word bit comes from(Binary digih because zeros and 

ones are the digits used in binary representations of numbers. The well-known statistician John 

T 1 Tukey introduced this terminology in 1946. A bit can be used to represent a truth value, because 

F there are two truth values, namely, true and false. As is customarily done, we will use a | bit to 

represent true and a 0 bit to represent false. That is, 1 represents T (true), 0 represents F (false). A 

variable is called a Boolean variable if its value is either true or false. Consequently, a Boolean 
variable can be represented using a bit. 

Computer bit operations correspond to the logical connectives. By replacing true by a one 

and false by a zero in the truth tables for the operators A, V, and ®, the tables shown in Table 9 

for the corresponding bit operations are obtained. We will also use the notation 


JOHN WILDER TUKEY (1915-2000) Tukey, born in New Bedford, Massachusetts, was an only child. His 
parents, both teachers, decided home schooling would best develop his potential. His formal education began 
at Brown University, where he studied mathematics and chemistry. He received a master’s degree in chemistry 
from Brown and continued his studies at Princeton University, changing his field of study from chemistry to 
mathematics. He received his Ph.D. from Princeton in 1939 for work in topology, when he was appointed an 
instructor in mathematics at Princeton. With the start of World War II, he joined the Fire Control Research Office, 
where he began working in statistics. Tukey found statistical research to his liking and impressed several leading 
statisticians with his skills. In 1945, at the conclusion of the war, Tukey returned to the mathematics department 
at Princeton as a professor of statistics, and he also took a position at AT&T Bell Laboratories. Tukey founded 
the Statistics Department at Princeton in 1966 and was its first chairman. Tukey made significant contributions to many areas of 
statistics, including the analysis of variance, the estimation of spectra of time series, inferences about the values of a set of parameters 
from a single experiment, and the philosophy of statistics. However, he is best known for his invention, with J. W. Cooley, of the fast 
Fourier transform. In addition to his contributions to statistics, Tukey was noted as a skilled wordsmith; he is credited with coining 
the terms bit and software. 

Tukey contributed his insight and expertise by serving on the President’s Science Advisory Committee. He chaired several 
important committees dealing with the environment, education, and chemicals and health. He also served on committees working 
on nuclear disarmament. Tukey received many awards, including the National Medal of Science. 


HISTORICAL NOTE There were several other suggested words for a binary digit, including binit and bigit, that never were widely 
accepted. The adoption of the word bit may be due to its meaning as a common English word. For an account of Tukey’s coining 
of the word bit, see the April 1984 issue of Annals of the History of Computing. 
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TABLE 9 Table for the Bit Operators OR, 
AND, and XOR. 


Information is often represented using bit strings, which are lists of zeros and ones. When 
this is done, operations on the bit strings can be used to manipulate this information. 


DEFINITION 7 _ A bit string is a sequence of zero or more bits. The /ength of this string is the number of bits 
in the string. 

EXAMPLE 12 101010011 is a bit string of length nine. < 

We can extend bit operations to bit strings. We define the bitwise OR, bitwise AND, and 
bitwise XOR of two strings of the same length to be the strings that have as their bits the OR, 
AND, and XOR of the corresponding bits in the two strings, respectively. We use the symbols 
V, A, and @ to represent the bitwise OR, bitwise AND, and bitwise XOR operations, respectively. 
We illustrate bitwise operations on bit strings with Example 13. 

EXAMPLE 13 Find the bitwise OR, bitwise AND, and bitwise XOR of the bit strings 0110110110 and 
1100011101. (Here, and throughout this book, bit strings will be split into blocks of four 
bits to make them easier to read.) 

Solution: The bitwise OR, bitwise AND, and bitwise XOR of these strings are obtained by taking 
the OR, AND, and XOR of the corresponding bits, respectively. This gives us 
O1 1011 0110 
11 0001 1101 
11 1011 1111 bitwise OR 
01 0001 0100 bitwise AND 
10 1010 1011 bitwise XOR 4 
Exercises 
1. Which of these sentences are propositions? What are the d) 4+x=5. 
truth values of those that are propositions? e) The moon is made of green cheese. 
a) Boston is the capital of Massachusetts. f) 2" > 100. 


b) Miami is the capital of Florida. 


ec) 24+3=5. 
d) 5+7= 10. 
e) x+2=11. 


f) Answer this question. 


. Which of these are propositions? What are the truth values 


of those that are propositions? 

a) Do not pass go. 

b) What time is it? 

c) There are no black flies in Maine. 


3. What is the negation of each of these propositions? 
a) Mei has an MP3 player. 
b) There is no pollution in New Jersey. 
ce) 24+1=3. 
d) The summer in Maine is hot and sunny. 
4. What is the negation of each of these propositions? 
a) Jennifer and Teja are friends. 
b) There are 13 items in a baker’s dozen. 
c) Abby sent more than 100 text messages every day. 
d) 121 is a perfect square. 


5. What is the negation of each of these propositions? 


a) Steve has more than 100 GB free disk space on his 
laptop. 

b) Zach blocks e-mails and texts from Jennifer. 

ce) 7-11-13 = 999. 

d) Diane rode her bicycle 100 miles on Sunday. 

. Suppose that Smartphone A has 256 MB RAM and 32 GB 

ROM, and the resolution of its camera is 8 MP; Smart- 

phone B has 288 MB RAM and 64 GB ROM, and the 

resolution of its camera is 4 MP; and Smartphone C has 

128 MB RAM and 32 GB ROM, and the resolution of 

its camera is 5 MP. Determine the truth value of each of 

these propositions. 

a) Smartphone B has the most RAM of these three smart- 
phones. 

b) Smartphone C has more ROM or a higher resolution 
camera than Smartphone B. 

c) Smartphone B has more RAM, more ROM, and a 
higher resolution camera than Smartphone A. 

d) If Smartphone B has more RAM and more ROM than 
Smartphone C, then it also has a higher resolution 
camera. 

e) Smartphone A has more RAM than Smartphone B if 
and only if Smartphone B has more RAM than Smart- 
phone A. 

. Suppose that during the most recent fiscal year, the an- 

nual revenue of Acme Computer was 138 billion dollars 

and its net profit was 8 billion dollars, the annual revenue 
of Nadir Software was 87 billion dollars and its net profit 
was 5 billion dollars, and the annual revenue of Quixote 

Media was 111 billion dollars and its net profit was 

13 billion dollars. Determine the truth value of each of 

these propositions for the most recent fiscal year. 

a) Quixote Media had the largest annual revenue. 

b) Nadir Software had the lowest net profit and Acme 
Computer had the largest annual revenue. 

c) Acme Computer had the largest net profit or Quixote 
Media had the largest net profit. 

d) If Quixote Media had the smallest net profit, then 
Acme Computer had the largest annual revenue. 

e) Nadir Software had the smallest net profit if and only 
if Acme Computer had the largest annual revenue. 


. Let p and q be the propositions 

p : I bought a lottery ticket this week. 

q : | won the million dollar jackpot. 
Express each of these propositions as an English sen- 
tence. 


a) —p b) pvq ec) p>q 
d) pq e) pg f) ~p> —q 
g) =pA~q h) =pv (pA q) 


. Let p and q be the propositions “Swimming at the New 
Jersey shore is allowed” and “Sharks have been spotted 
near the shore,” respectively. Express each of these com- 
pound propositions as an English sentence. 


a) 7q b) pAq c) =pvq 
d) p> ~q e) -q> p f) -p>-q 
g) pg h) =p) (pv —@) 


10. 


11. 


12. 


13. 


14. 


1.1 Propositional Logic 13 


Let p and q be the propositions “The election is decided” 
and “The votes have been counted,” respectively. Express 
each of these compound propositions as an English sen- 
tence. 


a) —p b) pvq 

©) =pAq d)q->p 

e) -q> 7p f) ~p> —q 
g)p<ogd h) -qV (7pA q) 


Let p and q be the propositions 
p : It is below freezing. 
q : Itis snowing. 
Write these propositions using p and g and logical con- 
nectives (including negations). 
a) It is below freezing and snowing. 
b) It is below freezing but not snowing. 
c) It is not below freezing and it is not snowing. 
d) It is either snowing or below freezing (or both). 
e) If itis below freezing, it is also snowing. 
f) Either it is below freezing or it is snowing, but it is 
not snowing if it is below freezing. 
g) That it is below freezing is necessary and sufficient 
for it to be snowing. 
Let p,q, and r be the propositions 
p : You have the flu. 
q : You miss the final examination. 
r : You pass the course. 
Express each of these propositions as an English sen- 
tence. 
a) pq 
ce) q->7>r 
e) (p> -r)VqG-> -r) 
f) PAQVCaAr) 
Let p and q be the propositions 
p : You drive over 65 miles per hour. 
q : You get a speeding ticket. 
Write these propositions using p and q and logical con- 
nectives (including negations). 
a) You do not drive over 65 miles per hour. 
b) You drive over 65 miles per hour, but you do not get 
a speeding ticket. 
c) You will get a speeding ticket if you drive over 
65 miles per hour. 
d) If you do not drive over 65 miles per hour, then you 
will not get a speeding ticket. 
e) Driving over 65 miles per hour is sufficient for getting 
a speeding ticket. 
f) You get a speeding ticket, but you do not drive over 
65 miles per hour. 
g) Whenever you get a speeding ticket, you are driving 
over 65 miles per hour. 


b) -qeor 
d) pVqvr 


Let p,q, and r be the propositions 

p : You get an A on the final exam. 

q : You do every exercise in this book. 

r : You get an A in this class. 
Write these propositions using p, qg, and r and logical 
connectives (including negations). 
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17. 


18. 
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a) You get an A in this class, but you do not do every 
exercise in this book. 

b) You get an A on the final, you do every exercise in this 
book, and you get an A in this class. 

c) To get an A in this class, it is necessary for you to get 
an A on the final. 

d) You get an A on the final, but you don’t do every ex- 
ercise in this book; nevertheless, you get an A in this 
class. 

e) Getting an A on the final and doing every exercise in 
this book is sufficient for getting an A in this class. 

f) You will get an A in this class if and only if you either 
do every exercise in this book or you get an A on the 
final. 


Let p,q, and r be the propositions 


p : Grizzly bears have been seen in the area. 
q : Hiking is safe on the trail. 
r : Berries are ripe along the trail. 

Write these propositions using p, qg, and r and logical 

connectives (including negations). 

a) Berries are ripe along the trail, but grizzly bears have 
not been seen in the area. 

b) Grizzly bears have not been seen in the area and hik- 
ing on the trail is safe, but berries are ripe along the 
trail. 

c) If berries are ripe along the trail, hiking is safe if and 
only if grizzly bears have not been seen in the area. 

d) Itis not safe to hike on the trail, but grizzly bears have 
not been seen in the area and the berries along the trail 
are ripe. 

e) For hiking on the trail to be safe, it is necessary but not 
sufficient that berries not be ripe along the trail and 
for grizzly bears not to have been seen in the area. 

f) Hiking is not safe on the trail whenever grizzly bears 
have been seen in the area and berries are ripe along 
the trail. 

Determine whether these biconditionals are true or 

false. 

a) 2+2=4ifand only if1+1=2. 

b) 1+ 1 =2 if and only if2+3=4. 

c) 1+ 1 =3 if and only if monkeys can fly. 

d) 0 > 1ifand only if 2 > 1. 

Determine whether each of these conditional statements 

is true or false. 

a) Ifl1+1=2,then2+2=5. 

b) If 14+ 1=3,then2+2=4. 

ce) Ifl1+1=3,then2+2=5. 

d) If monkeys can fly, then 1 + 1 = 3. 

Determine whether each of these conditional statements 

is true or false. 


a) If 1 + 1 =3, then unicorns exist. 

b) If 1+ 1 = 3, then dogs can fly. 

ce) If 1+ 1 = 2, then dogs can fly. 

d) If2+2=4,then1+2=3. 

For each of these sentences, determine whether an in- 
clusive or, or an exclusive or, is intended. Explain your 
answer. 
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a) Coffee or tea comes with dinner. 

b) A password must have at least three digits or be at 
least eight characters long. 

c) The prerequisite for the course is a course in number 
theory or a course in cryptography. 

d) You can pay using U.S. dollars or euros. 

For each of these sentences, determine whether an in- 

clusive or, or an exclusive or, is intended. Explain your 

answer. 

a) Experience with C++ or Java is required. 

b) Lunch includes soup or salad. 

c) To enter the country you need a passport or a voter 
registration card. 

d) Publish or perish. 

For each of these sentences, state what the sentence means 

if the logical connective or is an inclusive or (that is, a dis- 

junction) versus an exclusive or. Which of these meanings 

of or do you think is intended? 

a) To take discrete mathematics, you must have taken 
calculus or a course in computer science. 

b) When you buy a new car from Acme Motor Company, 
you get $2000 back in cash or a 2% car loan. 

c) Dinner for two includes two items from column A or 
three items from column B. 

d) School is closed if more than 2 feet of snow falls or if 
the wind chill is below —100. 

Write each of these statements in the form “if p, then g” 

in English. [Hint: Refer to the list of common ways to ex- 

press conditional statements provided in this section.] 

a) Itis necessary to wash the boss’s car to get promoted. 

b) Winds from the south imply a spring thaw. 

c) A sufficient condition for the warranty to be good is 
that you bought the computer less than a year ago. 

d) Willy gets caught whenever he cheats. 

e) You can access the website only if you pay a subscrip- 
tion fee. 

f) Getting elected follows from knowing the right peo- 
ple. 

g) Carol gets seasick whenever she is on a boat. 

Write each of these statements in the form “if p, then g” 

in English. [Hint: Refer to the list of common ways to 

express conditional statements. ] 

a) Itsnows whenever the wind blows from the northeast. 

b) The apple trees will bloom if it stays warm for a week. 

c) That the Pistons win the championship implies that 
they beat the Lakers. 

d) It is necessary to walk 8 miles to get to the top of 
Long’s Peak. 

e) To get tenure as a professor, it is sufficient to be world- 
famous. 

f) Ifyou drive more than 400 miles, you will need to buy 
gasoline. 

g) Your guarantee is good only if you bought your CD 
player less than 90 days ago. 

h) Jan will go swimming unless the water is too cold. 
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Write each of these statements in the form “if p, then g” 
in English. [Hint: Refer to the list of common ways to ex- 
press conditional statements provided in this section.] 


a) I will remember to send you the address only if you 
send me an e-mail message. 

b) To be a citizen of this country, it is sufficient that you 
were born in the United States. 

c) Ifyou keep your textbook, it will be a useful reference 
in your future courses. 

d) The Red Wings will win the Stanley Cup if their goalie 
plays well. 

e) That you get the job implies that you had the best 
credentials. 

f) The beach erodes whenever there is a storm. 

g) It is necessary to have a valid password to log on to 
the server. 

h) You willreach the summit unless you begin your climb 
too late. 


Write each of these propositions in the form “p if and 

only if g” in English. 

a) If it is hot outside you buy an ice cream cone, and if 
you buy an ice cream cone it is hot outside. 

b) For you to win the contest it is necessary and sufficient 
that you have the only winning ticket. 

c) You get promoted only if you have connections, and 
you have connections only if you get promoted. 

d) Ifyou watch television your mind will decay, and con- 
versely. 

e) The trains run late on exactly those days when I take 
it. 

Write each of these propositions in the form “p if and 

only if g” in English. 

a) For you to get an A in this course, it is necessary and 
sufficient that you learn how to solve discrete mathe- 
matics problems. 

b) If you read the newspaper every day, you will be in- 
formed, and conversely. 

c) It rains if it is a weekend day, and it is a weekend day 
if it rains. 

d) You can see the wizard only if the wizard is not in, 
and the wizard is not in only if you can see him. 

State the converse, contrapositive, and inverse of each of 

these conditional statements. 


a) If it snows today, I will ski tomorrow. 

b) Icome to class whenever there is going to be a quiz. 

c) A positive integer is a prime only if it has no divisors 
other than | and itself. 

State the converse, contrapositive, and inverse of each of 

these conditional statements. 

a) If it snows tonight, then I will stay at home. 

b) I go to the beach whenever it is a sunny summer day. 

c) When I stay up late, it is necessary that I sleep until 
noon. 

How many rows appear in a truth table for each of these 

compound propositions? 


a) p> 7p 
b) (pV tr) A |v 7s) 
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ce) gqVpVr>sV-7rv-tvu 

d) (pPArTAtThSe (At) 

How many rows appear in a truth table for each of these 
compound propositions? 

a) (q> 7p) Vv (7p > 74) 

b) (pv -t) A (pv 7s) 

ce) (po rn)V (Cs > At) V (Fu => Vv) 

d) (PATAS)V(GAthV FARM) 

Construct a truth table for each of these compound propo- 
sitions. 

a) pAn-p b) pV 7p 

ce) (pV-q)> 4 d) (pV q) > (PAQq) 

e) (p> goCq> 7p) 

f) p> g)-> a> Pp) 

Construct a truth table for each of these compound propo- 
sitions. 

a) p> —p 

c) p®(pv gq) 

e) q> 7p (P<gq) 
f) (poqge(pe ag) 
Construct a truth table for each of these compound propo- 
sitions. 

a) (pV q) > (Pp ®q) 

c) (pV gq) ®(pAq) 

e) (Po gempc-r) 
f) (p®q) > (p®-9q) 
Construct a truth table for each of these compound propo- 
sitions. 
a) p@®p 


b) pp 
d) (pAq) > (PVQq) 


b) (pq) > (PAQ) 
d) (poqge(p< q) 


b) p®-p 

c) p@-|q d) —p®-q 

e) (POqQV(pe-gq) ff) (POq)APO-g) 
Construct a truth table for each of these compound propo- 
sitions. 

a) p> -q b) -poq 

e) (pP>gavcp>q dd PprgarAcPp- 4) 
e) (poq)v (“p+ q) 

f) (“po-g) <3 (pq) 

Construct a truth table for each of these compound propo- 
sitions. 

a) (pv q)vr 


b) (py q)Ar 

c) (pAqvr d) (pAqgAr 

e) (pV q)A7r f) (pAqg)Vorr 
Construct a truth table for each of these compound propo- 
sitions. 

a) p> (“GV r) 

b) -p>@->r) 

ce) (P>QgvCcp>r) 

d) p> QgACP>?r) 

e) (pogvcrqer) 

f) (pe-gec@<r) 

Construct a truth table for ((p > gq) > r) > s. 


Construct a truth table for (p <> gq) @ (r = 5S). 
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. Explain, without using a truth table, why (p Vv -=q) A 
(q V =r) A (r V ap) is true when p, qg, and r have the 
same truth value and it is false otherwise. 

Explain, without using a truth table, why (p Vq Vr) A 
(4p V -q V —r) is true when at least one of p, g, andr 
is true and at least one is false, but is false when all three 
variables have the same truth value. 

What is the value of x after each of these statements is 
encountered in a computer program, if x = 1 before the 
statement is reached? 

a) ifx+2=3thenx :=x+1 

b) if (x + 1 =3) OR (2x +2 = 3) thenx :=x+1 

ce) if (2x +3=5)AND Bx +4=7) thenx :=x+1 
d) if (x + 1 = 2) XOR (x +2 = 3) thenx :=x+1 

e) ifx <2thenx:=x+1 

Find the bitwise OR, bitwise AND, and bitwise XOR of 
each of these pairs of bit strings. 

a) 101 1110, 010 0001 

b) 1111 0000, 1010 1010 

c) 000111 0001, 100100 1000 

d) 11 1111 1111, 00 0000 0000 

Evaluate each of these expressions. 

a) 11000 A (0 1011 v 1 1011) 

b) (01111 A 1 0101) v 0 1000 

c) (01010 6 1 1011) 6 0 1000 

d) (1 1011 v0 1010) A (1 0001 v 1 1011) 


Fuzzy logic is used in artificial intelligence. In fuzzy logic, a 
proposition has a truth value that is a number between 0 and 1, 
inclusive. A proposition with a truth value of 0 is false and one 


wi 


th a truth value of 1 is true. Truth values that are between 0 


and 1 indicate varying degrees of truth. For instance, the truth 
value 0.8 can be assigned to the statement “Fred is happy,” 


because Fred is happy most of the time, and the truth value 
0.4 can be assigned to the statement “John is happy,” because 
John is happy slightly less than half the time. Use these truth 
values to solve Exercises 45-47. 
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The truth value of the negation of a proposition in fuzzy 
logic is 1 minus the truth value of the proposition. What 
are the truth values of the statements “Fred is not happy” 
and “John is not happy?” 

The truth value of the conjunction of two propositions in 
fuzzy logic is the minimum of the truth values of the two 
propositions. What are the truth values of the statements 
“Fred and John are happy” and “Neither Fred nor John is 
happy?” 

The truth value of the disjunction of two propositions in 
fuzzy logic is the maximum of the truth values of the two 
propositions. What are the truth values of the statements 
“Fred is happy, or John is happy” and “Fred is not happy, 
or John is not happy?” 


Is the assertion “This statement is false” a proposition? 


The nth statement in a list of 100 statements is “Exactly 
n of the statements in this list are false.” 


a) What conclusions can you draw from these state- 
ments? 

b) Answer part (a) if the nth statement is “At least n of 
the statements in this list are false.” 

c) Answer part (b) assuming that the list contains 99 
statements. 

An ancient Sicilian legend says that the barber in a remote 

town who can be reached only by traveling a dangerous 

mountain road shaves those people, and only those peo- 

ple, who do not shave themselves. Can there be such a 

barber? 


Applications of Propositional Logic 


Introduction 


Logic has many important applications to mathematics, computer science, and numerous other 
disciplines. Statements in mathematics and the sciences and in natural language often are im- 
precise or ambiguous. To make such statements precise, they can be translated into the language 
of logic. For example, logic is used in the specification of software and hardware, because these 
specifications need to be precise before development begins. Furthermore, propositional logic 
and its rules can be used to design computer circuits, to construct computer programs, to verify 
the correctness of programs, and to build expert systems. Logic can be used to analyze and 
solve many familiar puzzles. Software systems based on the rules of logic have been developed 
for constructing some, but not all, types of proofs automatically. We will discuss some of these 
applications of propositional logic in this section and in later chapters. 


Translating English Sentences 


There are many reasons to translate English sentences into expressions involving propositional 
variables and logical connectives. In particular, English (and every other human language) is 


EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


“ 


‘te 
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often ambiguous. Translating sentences into compound statements (and other types of logical 
expressions, which we will introduce later in this chapter) removes the ambiguity. Note that 
this may involve making a set of reasonable assumptions based on the intended meaning of the 
sentence. Moreover, once we have translated sentences from English into logical expressions 
we can analyze these logical expressions to determine their truth values, we can manipulate 
them, and we can use rules of inference (which are discussed in Section 1.6) to reason about 
them. 

To illustrate the process of translating an English sentence into a logical expression, consider 
Examples | and 2. 


How can this English sentence be translated into a logical expression? 


“You can access the Internet from campus only if you are a computer science major or you 
are not a freshman.” 


Solution: There are many ways to translate this sentence into a logical expression. Although it is 
possible to represent the sentence by a single propositional variable, such as p, this would not be 
useful when analyzing its meaning or reasoning with it. Instead, we will use propositional vari- 
ables to represent each sentence part and determine the appropriate logical connectives between 
them. In particular, we let a, c, and f represent “You can access the Internet from campus,” 
“You are a computer science major,’ and “You are a freshman,” respectively. Noting that “only 
if” is one way a conditional statement can be expressed, this sentence can be represented as 


a> (cv—-f). < 


How can this English sentence be translated into a logical expression? 


“You cannot ride the roller coaster if you are under 4 feet tall unless you are older than 16 
years old.” 


Solution: Let q, r, and s represent “You can ride the roller coaster,’ “You are under 4 feet tall,” 
and “You are older than 16 years old,” respectively. Then the sentence can be translated to 


(7, A 7s) > 7q. 


Of course, there are other ways to represent the original sentence as a logical expression, 
but the one we have used should meet our needs. < 


System Specifications 


Translating sentences in natural language (such as English) into logical expressions is an essential 
part of specifying both hardware and software systems. System and software engineers take 
requirements in natural language and produce precise and unambiguous specifications that can 
be used as the basis for system development. Example 3 shows how compound propositions 
can be used in this process. 


Express the specification “The automated reply cannot be sent when the file system is full” 
using logical connectives. 


Solution: One way to translate this is to let p denote “The automated reply can be sent” and 
q denote “The file system is full”’ Then —p represents “It is not the case that the automated 
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EXAMPLE 4 


EXAMPLE 5 


Links 


EXAMPLE 6 


Extra 
Examples 


@ 


reply can be sent,” which can also be expressed as “The automated reply cannot be sent.” 
Consequently, our specification can be represented by the conditional statementg > =p. 4 


(Fequirements that could be used to derive a contradiction. When specifications are not consistent, 


there would be no way to develop a system that satisfies all specifications. 


Determine whether these system specifications are consistent: 


“The diagnostic message is stored in the buffer or it is retransmitted.” 
“The diagnostic message is not stored in the buffer.” 
“Tf the diagnostic message is stored in the buffer, then it is retransmitted.” 


Solution: To determine whether these specifications are consistent, we first express them using 
logical expressions. Let p denote “The diagnostic message is stored in the buffer” and let g 
denote “The diagnostic message is retransmitted.” The specifications can then be written as 
DY q,—p, and p — q. An assignment of truth values that makes all three specifications true 
must have p false to make —p true. Because we want p V q to be true but p must be false, 
q must be true. Because p — q is true when p is false and q is true, we conclude that these 
specifications are consistent, because they are all true when p is false and q is true. We could 
come to the same conclusion by use of a truth table to examine the four possible assignments 
of truth values to p and q. < 


Do the system specifications in Example 4 remain consistent if the specification “The diagnostic 
message is not retransmitted” is added? 


Solution: By the reasoning in Example 4, the three specifications from that example are true 
only in the case when p is false and gq is true. However, this new specification is —g, which is 
false when q is true. Consequently, these four specifications are inconsistent. < 


Logical connectives are used extensively in searches of large collections of information, such 
as indexes of Web pages. Because these searches employ techniques from propositional logic, 
they are called Boolean searches. 

In Boolean searches, the connective AND is used to match records that contain both of 
two search terms, the connective OR is used to match one or both of two search terms, and the 
connective NOT (sometimes written as AND NOT ) is used to exclude a particular search term. 
Careful planning of how logical connectives are used is often required when Boolean searches 
are used to locate information of potential interest. Example 6 illustrates how Boolean searches 
are carried out. 


Web Page Searching Most Web search engines support Boolean searching techniques, which 
usually can help find Web pages about particular subjects. For instance, using Boolean searching 
to find Web pages about universities in New Mexico, we can look for pages matching NEW 
AND MEXICO AND UNIVERSITIES. The results of this search will include those pages that 
contain the three words NEW, MEXICO, and UNIVERSITIES. This will include all of the 
pages of interest, together with others such as a page about new universities in Mexico. (Note 
that in Google, and many other search engines, the word “AND” is not needed, although it is 
understood, because all search terms are included by default. These search engines also support 
the use of quotation marks to search for specific phrases. So, it may be more effective to search 
for pages matching “New Mexico” AND UNIVERSITIES.) 
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Next, to find pages that deal with universities in New Mexico or Arizona, we can search 
for pages matching (NEW AND MEXICO OR ARIZONA) AND UNIVERSITIES. (Note: Here 
the AND operator takes precedence over the OR operator. Also, in Google, the terms used for 
this search would be NEW MEXICO OR ARIZONA.) The results of this search will include 
all pages that contain the word UNIVERSITIES and either both the words NEW and MEXICO 
or the word ARIZONA. Again, pages besides those of interest will be listed. Finally, to find 
Web pages that deal with universities in Mexico (and not New Mexico), we might first look 
for pages matching MEXICO AND UNIVERSITIES, but because the results of this search will 
include pages about universities in New Mexico, as well as universities in Mexico, it might be 
better to search for pages matching (MEXICO AND UNIVERSITIES) NOT NEW. The results 
of this search include pages that contain both the words MEXICO and UNIVERSITIES but 
do not contain the word NEW. (In Google, and many other search engines, the word “NOT” is 
replaced by the symbol “-”. In Google, the terms used for this last search would be MEXICO 
UNIVERSITIES -NEW.) < 


Logic Puzzles 


Puzzles that can be solved using logical reasoning are known as logic puzzles. Solving logic 
Links Wid puzzles is an excellent way to practice working with the rules of logic. Also, computer programs 
designed to carry out logical reasoning often use well-known logic puzzles to illustrate their 
capabilities. Many people enjoy solving logic puzzles, published in periodicals, books, and on 
the Web, as a recreational activity. 

We will discuss two logic puzzles here. We begin with a puzzle originally posed by Raymond 
Smullyan, a master of logic puzzles, who has published more than a dozen books containing 
challenging puzzles that involve logical reasoning. In Section 1.3 we will also discuss the 

extremely popular logic puzzle Sudoku. 


EXAMPLE 7 _ In [Sm78] Smullyan posed many puzzles about an island that has two kinds of inhabitants, 
knights, who always tell the truth, and their opposites, knaves, who always lie. You encounter 


Extra p> 
Examples two people A and B. What are A and B if A says “B is a knight” and B says “The two of us are 
opposite types?” 


Solution: Let p and q be the statements that A is a knight and B is a knight, respectively, so that 
—p and —q are the statements that A is a knave and B is a knave, respectively. 

We first consider the possibility that A is a knight; this is the statement that p is true. If A is 
a knight, then he is telling the truth when he says that B is a knight, so that qg is true, and A and B 
are the same type. However, if B is a knight, then B’s statement that A and B are of opposite 
types, the statement (p A —q) V (=p Aq), would have to be true, which it is not, because A 
and B are both knights. Consequently, we can conclude that A is not a knight, that is, that p is 
false. 

If A is a knave, then because everything a knave says is false, A’s statement that B is 
a knight, that is, that g is true, is a lie. This means that g is false and B is also a knave. 
Furthermore, if B is a knave, then B’s statement that A and B are opposite types is a lie, 
which is consistent with both A and B being knaves. We can conclude that both A and B are 
knaves. < 


We pose more of Smullyan’s puzzles about knights and knaves in Exercises 19-23. In 
Exercises 24—31 we introduce related puzzles where we have three types of people, knights and 
knaves as in this puzzle together with spies who can lie. 

Next, we pose a puzzle known as the muddy children puzzle for the case of two children. 
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EXAMPLE 8 


In Chapter 12 we design 
some useful circuits. 


Links 


A father tells his two children, a boy and a girl, to play in their backyard without getting dirty. 
However, while playing, both children get mud on their foreheads. When the children stop 
playing, the father says “At least one of you has a muddy forehead,” and then asks the children 
to answer “Yes” or “No” to the question: “Do you know whether you have a muddy forehead?” 
The father asks this question twice. What will the children answer each time this question is 
asked, assuming that a child can see whether his or her sibling has a muddy forehead, but cannot 
see his or her own forehead? Assume that both children are honest and that the children answer 
each question simultaneously. 


Solution: Let s be the statement that the son has a muddy forehead and let d be the statement that 
the daughter has a muddy forehead. When the father says that at least one of the two children 
has a muddy forehead, he is stating that the disjunction s V d is true. Both children will answer 
“No” the first time the question is asked because each sees mud on the other child’s forehead. 
That is, the son knows that d is true, but does not know whether s is true, and the daughter 
knows that s is true, but does not know whether d is true. 

After the son has answered “No” to the first question, the daughter can determine that d 
must be true. This follows because when the first question is asked, the son knows that s Vv d is 
true, but cannot determine whether s is true. Using this information, the daughter can conclude 
that d must be true, for if d were false, the son could have reasoned that because s V d is true, 
then s must be true, and he would have answered “Yes” to the first question. The son can reason 
in a similar way to determine that s must be true. It follows that both children answer “Yes” the 
second time the question is asked. < 


Logic Circuits 


Propositional logic can be applied to the design of computer hardware. This was first observed 
in 1938 by Claude Shannon in his MIT master’s thesis. In Chapter 12 we will study this topic 
in depth. (See that chapter for a biography of Shannon.) We give a brief introduction to this 
application here. 

A logic circuit (or digital circuit) receives input signals p;, p2,..., Dn, each a bit [either 
0 (off) or 1 (on)], and produces output signals 51, s2,..., 5,, each a bit. In this section we will 
restrict our attention to logic circuits with a single output signal; in general, digital circuits may 
have multiple outputs. 


RAYMOND SMULLYAN (BORN 1919) Raymond Smullyan dropped out of high school. He wanted to study 
what he was really interested in and not standard high school material. After jumping from one university to 
the next, he earned an undergraduate degree in mathematics at the University of Chicago in 1955. He paid 
his college expenses by performing magic tricks at parties and clubs. He obtained a Ph.D. in logic in 1959 at 
Princeton, studying under Alonzo Church. After graduating from Princeton, he taught mathematics and logic at 
Dartmouth College, Princeton University, Yeshiva University, and the City University of New York. He joined 
the philosophy department at Indiana University in 1981 where he is now an emeritus professor. 

Smullyan has written many books on recreational logic and mathematics, including Satan, Cantor, and 
Infinity; What Is the Name of This Book?; The Lady or the Tiger?; Alice in Puzzleland; To Mock a Mockingbird; 


Forever Undecided; and The Riddle of Scheherazade: Amazing Logic Puzzles, Ancient and Modern. Because his logic puzzles are 
challenging, entertaining, and thought-provoking, he is considered to be a modern-day Lewis Carroll. Smullyan has also written 
several books about the application of deductive logic to chess, three collections of philosophical essays and aphorisms, and several 
advanced books on mathematical logic and set theory. He is particularly interested in self-reference and has worked on extending 
some of Gédel’s results that show that it is impossible to write a computer program that can solve all mathematical problems. He is 
also particularly interested in explaining ideas from mathematical logic to the public. 

Smullyan is a talented musician and often plays piano with his wife, who is a concert-level pianist. Making telescopes is one 
of his hobbies. He is also interested in optics and stereo photography. He states “I’ve never had a conflict between teaching and 
research as some people do because when I’m teaching, I’m doing research.” Smullyan is the subject of a documentary short film 
entitled This Film Needs No Title. 


EXAMPLE 9 


EXAMPLE 10 
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FIGURE 1 Basic logic gates. 
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FIGURE 2. A combinatorial circuit. 


Complicated digital circuits can be constructed from three basic circuits, called gates, shown 
in Figure 1. The inverter, or NOT gate, takes an input bit p, and produces as output —p. The 
OR gate takes two input signals p and q, each a bit, and produces as output the signal p Vv q. 
Finally, the AND gate takes two input signals p and q, each a bit, and produces as output the 
signal p A q. We use combinations of these three basic gates to build more complicated circuits, 
such as that shown in Figure 2. 

Given a circuit built from the basic logic gates and the inputs to the circuit, we determine 
the output by tracing through the circuit, as Example 9 shows. 


Determine the output for the combinatorial circuit in Figure 2. 


Solution: In Figure 2 we display the output of each logic gate in the circuit. We see that the AND 
gate takes input of p and —g, the output of the inverter with input g, and produces p A -q. 
Next, we note that the OR gate takes input p A —q and —r, the output of the inverter with 
input r, and produces the final output (p A -qg) V —r. 


Suppose that we have a formula for the output of a digital circuit in terms of negations, 
disjunctions, and conjunctions. Then, we can systematically build a digital circuit with the 
desired output, as illustrated in Example 10. 


Build a digital circuit that produces the output (p V =r) A (=p V (¢ V 7=r)) when given input 
bits p, g, andr. 


Solution: To construct the desired circuit, we build separate circuits for p V —r and for =p V 
(q V =r) and combine them using an AND gate. To construct a circuit for p V —r, we use an 
inverter to produce —r from the input r. Then, we use an OR gate to combine p and —r. To 
build a circuit for —p V (¢ V =r), we first use an inverter to obtain ~r. Then we use an OR gate 
with inputs g and —r to obtain g V —r. Finally, we use another inverter and an OR gate to get 
=p V (q V =r) from the inputs p and gq v -r. 

To complete the construction, we employ a final AND gate, with inputs p V —=r and =p v 
(q V -r). The resulting circuit is displayed in Figure 3. < 


We will study logic circuits in great detail in Chapter 12 in the context of Boolean algebra, 
and with different notation. 
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FIGURE 3 The circuit for (p V ~r) A (—=p V (q V -=r)). 
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Exercises 


In Exercises 1-6, translate the given statement into proposi- 
tional logic using the propositions provided. 
1. You cannot edit a protected Wikipedia entry unless you 


are an administrator. Express your answer in terms of e: 
“You can edit a protected Wikipedia entry” and a: “You 
are an administrator.” 


2. You can see the movie only if you are over 18 years old 


3 


or you have the permission of a parent. Express your an- 
swer in terms of m: “You can see the movie,” e: “You are 
over 18 years old,” and p: “You have the permission of a 
parent.” 

You can graduate only if you have completed the require- 
ments of your major and you do not owe money to the 
university and you do not have an overdue library book. 
Express your answer in terms of g: “You can graduate,” 
m: “You owe money to the university,” 7: “You have com- 
pleted the requirements of your major,” and b: “You have 
an overdue library book.” 


4. To use the wireless network in the airport you must pay 


5 


6 


the daily fee unless you are a subscriber to the service. 
Express your answer in terms of w: “You can use the wire- 
less network in the airport,” d: “You pay the daily fee,” 
and s: “You are a subscriber to the service.” 


You are eligible to be President of the U.S.A. only if you 
are at least 35 years old, were born in the U.S.A, or at the 
time of your birth both of your parents were citizens, and 
you have lived at least 14 years in the country. Express 
your answer in terms of e: “You are eligible to be Pres- 
ident of the U.S.A.,” a: “You are at least 35 years old,” 
b: “You were born in the U.S.A,” p: “At the time of your 
birth, both of your parents where citizens,” and r: “You 
have lived at least 14 years in the U.S.A.” 


You can upgrade your operating system only if you have 
a 32-bit processor running at 1 GHz or faster, at least 
1 GB RAM, and 16 GB free hard disk space, or a 64- 
bit processor running at 2 GHz or faster, at least 2 GB 
RAM, and at least 32 GB free hard disk space. Express 
you answer in terms of uw: “You can upgrade your oper- 
ating system,” b39: “You have a 32-bit processor,” b64: 


9. 


“You have a 64-bit processor,” g1: “Your processor runs 
at 1 GHz or faster,” g2: “Your processor runs at 2 GHz or 
faster,” r,: “Your processor has at least 1 GB RAM,” ro: 
“Your processor has at least 2 GB RAM,” hyo: “You have 
at least 16 GB free hard disk space,” and 32: “You have 
at least 32 GB free hard disk space.” 


Express these system specifications using the proposi- 
tions p “The message is scanned for viruses” and q “The 
message was sent from an unknown system” together 
with logical connectives (including negations). 


a) “The message is scanned for viruses whenever the 
message was sent from an unknown system.” 


b) “The message was sent from an unknown system but 
it was not scanned for viruses.” 


c) “Itis necessary to scan the message for viruses when- 
ever it was sent from an unknown system.” 


d) “When a message is not sent from an unknown system 
it is not scanned for viruses.” 


Express these system specifications using the proposi- 
tions p “The user enters a valid password,” q “Access is 
granted,” and r “The user has paid the subscription fee” 
and logical connectives (including negations). 


a) “The user has paid the subscription fee, but does not 
enter a valid password.” 


b) “Access is granted whenever the user has paid the 
subscription fee and enters a valid password.” 


c) “Access is denied if the user has not paid the subscrip- 
tion fee.” 


d) “If the user has not entered a valid password but has 
paid the subscription fee, then access is granted.” 


Are these system specifications consistent? “The system 
is in multiuser state if and only if it is operating normally. 
If the system is operating normally, the kernel is func- 
tioning. The kernel is not functioning or the system is 
in interrupt mode. If the system is not in multiuser state, 
then it is in interrupt mode. The system is not in interrupt 
mode.” 


10. 


11. 


12. 


13. 


14. 


#15, 


16. 


17. 


18. 


Are these system specifications consistent? “Whenever 
the system software is being upgraded, users cannot ac- 
cess the file system. If users can access the file system, 
then they can save new files. If users cannot save new 
files, then the system software is not being upgraded.” 


Are these system specifications consistent? “The router 
can send packets to the edge system only if it supports the 
new address space. For the router to support the new ad- 
dress space it is necessary that the latest software release 
be installed. The router can send packets to the edge sys- 
tem if the latest software release is installed, The router 
does not support the new address space.” 


Are these system specifications consistent? “If the file 
system is not locked, then new messages will be queued. 
If the file system is not locked, then the system is func- 
tioning normally, and conversely. If new messages are not 
queued, then they will be sent to the message buffer. If 
the file system is not locked, then new messages will be 
sent to the message buffer. New messages will not be sent 
to the message buffer.” 


What Boolean search would you use to look for Web 
pages about beaches in New Jersey? What if you wanted 
to find Web pages about beaches on the isle of Jersey (in 
the English Channel)? 


What Boolean search would you use to look for Web 
pages about hiking in West Virginia? What if you wanted 
to find Web pages about hiking in Virginia, but not in West 
Virginia? 

Each inhabitant of a remote village always tells the truth 
or always lies. A villager will give only a “Yes” or a “No” 
response to a question a tourist asks. Suppose you are a 
tourist visiting this area and come to a fork in the road. 
One branch leads to the ruins you want to visit; the other 
branch leads deep into the jungle. A villager is standing 
at the fork in the road. What one question can you ask the 
villager to determine which branch to take? 


An explorer is captured by a group of cannibals. There are 
two types of cannibals—those who always tell the truth 
and those who always lie. The cannibals will barbecue 
the explorer unless he can determine whether a particu- 
lar cannibal always lies or always tells the truth. He is 
allowed to ask the cannibal exactly one question. 


a) Explain why the question “Are you a liar?” does not 
work. 

b) Find a question that the explorer can use to determine 
whether the cannibal always lies or always tells the 
truth. 


When three professors are seated in a restaurant, the host- 
ess asks them: “Does everyone want coffee?” The first 
professor says: “I do not know.” The second professor 
then says: “I do not know.” Finally, the third professor 
says: “No, not everyone wants coffee.” The hostess comes 
back and gives coffee to the professors who want it. How 
did she figure out who wanted coffee? 

When planning a party you want to know whom to in- 
vite. Among the people you would like to invite are three 
touchy friends. You know that if Jasmine attends, she will 
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become unhappy if Samir is there, Samir will attend only 
if Kanti will be there, and Kanti will not attend unless Jas- 
mine also does. Which combinations of these three friends 
can you invite so as not to make someone unhappy? 
Exercises 19—23 relate to inhabitants of the island of knights 
and knaves created by Smullyan, where knights always tell 
the truth and knaves always lie. You encounter two people, 
A and B. Determine, if possible, what A and B are if they 
address you in the ways described. If you cannot determine 
what these two people are, can you draw any conclusions? 


19. A says “At least one of us is a knave” and B says nothing. 


20. A says “The two of us are both knights” and B says “A 
is a knave.” 


21. A says “Iam aknave or B is aknight” and B says nothing. 
22. Both A and B say “I am a knight.” 


23. A says “We are both knaves” and B says nothing. 


Exercises 24—31 relate to inhabitants of an island on which 
there are three kinds of people: knights who always tell the 
truth, knaves who always lie, and spies (called normals by 
Smullyan [Sm78]) who can either lie or tell the truth. You 
encounter three people, A, B, and C. You know one of these 
people is a knight, one is a knave, and one is a spy. Each of the 
three people knows the type of person each of other two is. For 
each of these situations, if possible, determine whether there 
is a unique solution and determine who the knave, knight, and 
spy are. When there is no unique solution, list all possible 
solutions or state that there are no solutions. 


24. A says “C is the knave,” B says, “A is the knight,” and C 
says “I am the spy.” 


25. A says “I am the knight,’ B says “I am the knave,” and 
C says “B is the knight.” 

26. A says “I am the knave,” B says “I am the knave,” and C 
says “I am the knave.” 


27. A says “I am the knight,” B says “A is telling the truth,” 
and C says “I am the spy.” 


28. A says “I am the knight,” B says, “A is not the knave,” 
and C says “B is not the knave.” 


29. A says “I am the knight,” B says “I am the knight,’ and 
C says “I am the knight.” 


30. A says “I am not the spy,” B says “I am not the spy,” and 
C says “A is the spy.” 


31. A says “I am not the spy,” B says “I am not the spy,” and 
C says “I am not the spy.” 


Exercises 32-38 are puzzles that can be solved by translating 
statements into logical expressions and reasoning from these 
expressions using truth tables. 


32. The police have three suspects for the murder of Mr. 
Cooper: Mr. Smith, Mr. Jones, and Mr. Williams. Smith, 
Jones, and Williams each declare that they did not kill 
Cooper. Smith also states that Cooper was a friend of 
Jones and that Williams disliked him. Jones also states 
that he did not know Cooper and that he was out of town 
the day Cooper was killed. Williams also states that he 
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34. 


35. 


36. 


37. 


* 38. 
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saw both Smith and Jones with Cooper the day of the 
killing and that either Smith or Jones must have killed 
him. Can you determine who the murderer was if 


a) one of the three men is guilty, the two innocent men 
are telling the truth, but the statements of the guilty 
man may or may not be true? 

b) innocent men do not lie? 

Steve would like to determine the relative salaries of three 

coworkers using two facts. First, he knows that if Fred 

is not the highest paid of the three, then Janice is. Sec- 
ond, he knows that if Janice is not the lowest paid, then 

Maggie is paid the most. Is it possible to determine the 

relative salaries of Fred, Maggie, and Janice from what 

Steve knows? If so, who is paid the most and who the 

least? Explain your reasoning. 

Five friends have access to a chat room. Is it possible to 

determine who is chatting if the following information is 

known? Either Kevin or Heather, or both, are chatting. 

Either Randy or Vijay, but not both, are chatting. If Abby 

is chatting, so is Randy. Vijay and Kevin are either both 

chatting or neither is. If Heather is chatting, then so are 

Abby and Kevin. Explain your reasoning. 

A detective has interviewed four witnesses to a crime. 

From the stories of the witnesses the detective has con- 

cluded that if the butler is telling the truth then so is the 

cook; the cook and the gardener cannot both be telling the 
truth; the gardener and the handyman are not both lying; 
and if the handyman is telling the truth then the cook is 
lying. For each of the four witnesses, can the detective de- 
termine whether that person is telling the truth or lying? 

Explain your reasoning. 

Four friends have been identified as suspects for an unau- 

thorized access into a computer system. They have made 

statements to the investigating authorities. Alice said 

“Carlos did it.’ John said “I did not do it.’ Carlos said 

“Diana did it.” Diana said “Carlos lied when he said that 

I did it.” 

a) If the authorities also know that exactly one of the 
four suspects is telling the truth, who did it? Explain 


your reasoning. 
b) If the authorities also know that exactly one is lying, 


who did it? Explain your reasoning. 


Suppose there are signs on the doors to two rooms. The 
sign on the first door reads “In this room there is a lady, 
and in the other one there is a tiger”; and the sign on the 
second door reads “In one of these rooms, there is a lady, 
and in one of them there is a tiger.” Suppose that you 
know that one of these signs is true and the other is false. 
Behind which door is the lady? 

Solve this famous logic puzzle, attributed to Albert Ein- 
stein, and known as the zebra puzzle. Five men with 
different nationalities and with different jobs live in con- 
secutive houses on a street. These houses are painted dif- 
ferent colors. The men have different pets and have dif- 
ferent favorite drinks. Determine who owns a zebra and 
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40. 


41. 


42. 


43. 


whose favorite drink is mineral water (which is one of the 
favorite drinks) given these clues: The Englishman lives 
in the red house. The Spaniard owns a dog. The Japanese 
man is a painter. The Italian drinks tea. The Norwegian 
lives in the first house on the left. The green house is 
immediately to the right of the white one. The photogra- 
pher breeds snails. The diplomat lives in the yellow house. 
Milk is drunk in the middle house. The owner of the green 
house drinks coffee. The Norwegian’s house is next to the 
blue one. The violinist drinks orange juice. The fox is in 
a house next to that of the physician. The horse is in a 
house next to that of the diplomat. [Hint: Make a table 
where the rows represent the men and columns represent 
the color of their houses, their jobs, their pets, and their 
favorite drinks and use logical reasoning to determine the 
correct entries in the table. ] 

Freedonia has fifty senators. Each senator is either honest 
or corrupt. Suppose you know that at least one of the Free- 
donian senators is honest and that, given any two Free- 
donian senators, at least one is corrupt. Based on these 
facts, can you determine how many Freedonian senators 
are honest and how many are corrupt? If so, what is the 
answer? 


Find the output of each of these combinatorial circuits. 
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Find the output of each of these combinatorial circuits. 
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Construct a combinatorial circuit using inverters, 
OR gates, and AND gates that produces the output 
(p A-=r) V (=q Ar) from input bits p, g, and r. 
Construct a combinatorial circuit using inverters, 
OR gates, and AND gates that produces the output 
((=p V 7=r) A7qg) V (ap A (q V r)) from input bits p, 
g,andr. 
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Propositional Equivalences 


DEFINITION 1 


EXAMPLE 1 


Demo 


DEFINITION 2 


Introduction 


An important type of step used in a mathematical argument is the replacement of a statement 
with another statement with the same truth value. Because of this, methods that produce propo- 
sitions with the same truth value as a given compound proposition are used extensively in the 
construction of mathematical arguments. Note that we will use the term “compound proposi- 
tion” to refer to an expression formed from propositional variables using logical operators, such 
as p Aq. 

We begin our discussion with a classification of compound propositions according to their 
possible truth values. 


A compound proposition that is always true, no matter what the truth values of the proposi- 
tional variables that occur in it, is called a tautology. A compound proposition that is always 
false is called a contradiction. A compound proposition that is neither a tautology nor a 
contradiction is called a contingency. 


Tautologies and contradictions are often important in mathematical reasoning. Example | illus- 
trates these types of compound propositions. 


We can construct examples of tautologies and contradictions using just one propositional vari- 
able. Consider the truth tables of p V —p and p A =p, shown in Table 1. Because p V >: is 
always true, it is a tautology. Because p A —p is always false, it is a contradiction. 


Logical Equivalences 
Compound propositions that have the same truth values in all possible cases are called logically 


equivalent. We can also define this notion as follows. 


The compound propositions p and gq are called logically equivalent if p <> q is a tautology. 
The notation p = q denotes that p and gq are logically equivalent. 


Remark: The symbol = is not a logical connective, and p = q 1s not a compound proposition 
but rather is the statement that p <— q is a tautology. The symbol } is sometimes used instead 
of = to denote logical equivalence. 


One way to determine whether two compound propositions are equivalent is to use a truth 
table. In particular, the compound propositions p and q are equivalent if and only if the columns 


TABLE 1 Examples of a Tautology 
and a Contradiction. 
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Extra p> 
Examples 


EXAMPLE 2 


EXAMPLE 3 


TABLE 2 De 
Morgan’s Laws. 


(p \q) =7pv~q 


(pV q) =7pA~q 


giving their truth values agree. Example 2 illustrates this method to establish an extremely 
important and useful logical equivalence, namely, that of —(p V q) with =p A 7q. This logical 
equivalence is one of the two De Morgan laws, shown in Table 2, named after the English 
mathematician Augustus De Morgan, of the mid-nineteenth century. 


Show that =(p Vv q) and =p A 77g are logically equivalent. 
Solution: The truth tables for these compound propositions are displayed in Table 3. Because 
the truth values of the compound propositions =(p V gq) and =p A 7g agree for all possible 


combinations of the truth values of p and q, it follows that -(p V q) = (=p A 7q) isatautology 
and that these compound propositions are logically equivalent. 


TABLE 3 Truth Tables for =(p Vv q) and =p A -q. 
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Show that p — q and —p V g are logically equivalent. 


Solution: We construct the truth table for these compound propositions in Table 4. Because the 
truth values of =p Vv q and p > q agree, they are logically equivalent. < 


TABLE 4 Truth Tables for —p v q and 
p> 4: 


We will now establish a logical equivalence of two compound propositions involving three 
different propositional variables p, g, and r. To use a truth table to establish such a logical 
equivalence, we need eight rows, one for each possible combination of truth values of these 
three variables. We symbolically represent these combinations by listing the truth values of p, 
q, and r, respectively. These eight combinations of truth values are TTT, TTF, TFT, TFF, FTT, 
FTF, FFT, and FFF; we use this order when we display the rows of the truth table. Note that we 
need to double the number of rows in the truth tables we use to show that compound propositions 
are equivalent for each additional propositional variable, so that 16 rows are needed to establish 
the logical equivalence of two compound propositions involving four propositional variables, 
and so on. In general, 2” rows are required if a compound proposition involves n propositional 
variables. 
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TABLE 5 A Demonstration That p v (q A,r) and (p V q) A (p V r) Are Logically 
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EXAMPLE 4 Show that p v (q Ar) and (p V gq) A (p Vr) are logically equivalent. This is the distributive 
law of disjunction over conjunction. 


Solution: We construct the truth table for these compound propositions in Table 5. Because 
the truth values of p V (q Ar) and (p V q) A (p Vr) agree, these compound propositions are 
logically equivalent. < 


The identities in Table 6 3 Fi j 
stan gpeciabcammof Table 6 contains some important equivalences. In these equivalences, T denotes the com- 


Boolean algebra identities | Pound proposition that is always true and F denotes the compound proposition that is always 
found in Table 5 of 


Section 12.1. See Table 1 
in Section 2.2 for TABLE 6 Logical Equivalences. 
— Name 


PAT=D 
pVF=p 


DV p=pD Idempotent laws 
PA\Pp=pPp 


(7p) =p Double negation law 


PVG=4qVpP Commutative laws 
prAQgC=EChrp 


(pPVq)Vr=pV(qvVr) Associative laws 
(PAQ Ar=pPA(Ar) 
PV(GAr)=(PYQA(PVr) Distributive laws 
DAV Tr) =(PAQ)V (PAP) 


(pAqg)=7pVvn7-q De Morgan’s laws 


(pV q) = 7p Aq 


PV(PAQ=P Absorption laws 
DP\(PVG)=P 


Negation laws 


28 1/The Foundations: Logic and Proofs 


When using De Morgan’s 
laws, remember to change 
the logical connective 
after you negate. 


TABLE 7 Logical Equivalences 
Involving Conditional Statements. 


p>q=-PVv4q 
p>q=-q>—7p 
pVq=7p>4q 
pAqg=-(p > 79) 
“(p> 9) = pA~q 


TABLE 8 Logical 
Equivalences Involving 
Biconditional Statements. 


pPoq=(p>qAq- p) 
peq=r7pc7-q 


Pq >=(PAQYV (PA74q) 


(pegq=pcr7g 


(p> qQ)A(p>r)=p> qr) 


(prrnAqrarn=PpVqgrr 
(p> QV(por)=p> @qvr) 


(prrnVv(q>r=(pPAgror 


false. We also display some useful equivalences for compound propositions involving condi- 
tional statements and biconditional statements in Tables 7 and 8, respectively. The reader is 
asked to verify the equivalences in Tables 6-8 in the exercises. 

The associative law for disjunction shows that the expression p V q V r is well defined, 
in the sense that it does not matter whether we first take the disjunction of p with g and then 
the disjunction of p V qg with r, or if we first take the disjunction of g and r and then take the 
disjunction of p withg V r. Similarly, the expression p A q A r is well defined. By extending this 
reasoning, it follows that pj V p2 V--- V py, and pj A p2 A--++ A pp are well defined whenever 
P1; P2,--++ Pn are propositions. 

Furthermore, note that De Morgan’s laws extend to 


A(piV pr2V-++V Pn) = (Api Amp2 A-++Aapn) 


and 
(pi A p2 A+++ A Pn) = (A P1V mp2 V +++ pn) 


We will sometimes use the notation Vi=1 pj; for pi V p2V-""V Pn and Nj=1 pj; for 
Pi A p2 A-:-+A pn. Using this notation, the extended version of De Morgan’s laws can be 
written concisely as = Viet Pj) = si “Pj and —( N\j=1 pi)= Viet —p;. (Methods for 
proving these identities will be given in Section 5.1.) 


Using De Morgan’s Laws 


The two logical equivalences known as De Morgan’s laws are particularly important. They tell 
us how to negate conjunctions and how to negate disjunctions. In particular, the equivalence 
aA(pV q) = 7p A7¢ tells us that the negation of a disjunction is formed by taking the con- 
junction of the negations of the component propositions. Similarly, the equivalence =(p A q) = 
=p V 7@¢ tells us that the negation of a conjunction is formed by taking the disjunction of the 
negations of the component propositions. Example 5 illustrates the use of De Morgan’s laws. 
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EXAMPLE 5 _ Use De Morgan’s laws to express the negations of “Miguel has a cellphone and he has a laptop 
computer” and “Heather will go to the concert or Steve will go to the concert.” 


* Solution: Let p be “Miguel has a cellphone” and g be “Miguel has a laptop computer.” Then 
Assessment “Miguel has a cellphone and he has a laptop computer” can be represented by p A q. By the 
first of De Morgan’s laws, —(p A q) is equivalent to —p V —q. Consequently, we can express 
the negation of our original statement as “Miguel does not have a cellphone or he does not have 
a laptop computer.” 
Let r be “Heather will go to the concert” and s be “Steve will go to the concert.” Then 
“Heather will go to the concert or Steve will go to the concert” can be represented by r Vv s. 
By the second of De Morgan’s laws, —=(r V s) is equivalent to =r A ss. Consequently, we can 
express the negation of our original statement as “Heather will not go to the concert and Steve 
will not go to the concert.” < 


Constructing New Logical Equivalences 


The logical equivalences in Table 6, as well as any others that have been established (such as 
those shown in Tables 7 and 8), can be used to construct additional logical equivalences. The 
reason for this is that a proposition in a compound proposition can be replaced by a compound 
proposition that is logically equivalent to it without changing the truth value of the original 
compound proposition. This technique is illustrated in Examples 6-8, where we also use the 
fact that if p and q are logically equivalent and g and r are logically equivalent, then p and r 
are logically equivalent (see Exercise 56). 


EXAMPLE 6 Show that =(p > q) and p A 77 are logically equivalent. 


Solution: We could use a truth table to show that these compound propositions are equivalent 

Extra ~ (similar to what we did in Example 4). Indeed, it would not be hard to do so. However, we want 
Examples to illustrate how to use logical identities that we already know to establish new logical identities, 
something that is of practical importance for establishing equivalences of compound propositions 

with a large number of variables. So, we will establish this equivalence by developing a series of 


Links 


AUGUSTUS DE MORGAN (1806-1871) Augustus De Morgan was born in India, where his father was a 
colonel in the Indian army. De Morgan’s family moved to England when he was 7 months old. He attended 
private schools, where in his early teens he developed a strong interest in mathematics. De Morgan studied 
at Trinity College, Cambridge, graduating in 1827. Although he considered medicine or law, he decided on 
mathematics for his career. He won a position at University College, London, in 1828, but resigned after the 
college dismissed a fellow professor without giving reasons. However, he resumed this position in 1836 when 
his successor died, remaining until 1866. 

De Morgan was a noted teacher who stressed principles over techniques. His students included many famous 
mathematicians, including Augusta Ada, Countess of Lovelace, who was Charles Babbage’s collaborator in his 
work on computing machines (see page 31 for biographical notes on Augusta Ada). (De Morgan cautioned the countess against 
studying too much mathematics, because it might interfere with her childbearing abilities!) 

De Morgan was an extremely prolific writer, publishing more than 1000 articles in more than 15 periodicals. De Morgan also 
wrote textbooks on many subjects, including logic, probability, calculus, and algebra. In 1838 he presented what was perhaps the first 
clear explanation of an important proof technique known as mathematical induction (discussed in Section 5.1 of this text), a term 
he coined. In the 1840s De Morgan made fundamental contributions to the development of symbolic logic. He invented notations 
that helped him prove propositional equivalences, such as the laws that are named after him. In 1842 De Morgan presented what 
is considered to be the first precise definition of a limit and developed new tests for convergence of infinite series. De Morgan was 
also interested in the history of mathematics and wrote biographies of Newton and Halley. 

In 1837 De Morgan married Sophia Frend, who wrote his biography in 1882. De Morgan’s research, writing, and teaching left 
little time for his family or social life. Nevertheless, he was noted for his kindness, humor, and wide range of knowledge. 
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EXAMPLE 7 


EXAMPLE 8 


logical equivalences, using one of the equivalences in Table 6 at a time, starting with -(p —> q) 
and ending with p A —q. We have the following equivalences. 


“(p> q)=7Cpvag) by Example 3 
=-(4p) A-7q by the second De Morgan law 


= pA 7q by the double negation law 


Show that —=(p V (=p A q)) and =p A 7g are logically equivalent by developing a series of 
logical equivalences. 


Solution: We will use one of the equivalences in Table 6 at a time, starting with >(p V (=p A q)) 
and ending with sp A 7g. (Note: we could also easily establish this equivalence using a truth 
table.) We have the following equivalences. 


A(pV (=p Aq) =7pA7(-p Aq) by the second De Morgan law 
=-7p A [-(Cp)V 74g] by the first De Morgan law 
=—-pA(pv-7q) by the double negation law 
=(-pA p)V (=p A-7@) | by the second distributive law 
=Fv(-pA-q) because =p A p=F 
=(-=pA-7q)VF by the commutative law for disjunction 
= 7pA-q by the identity law for F 

Consequently =(p V (—=p A q)) and sp A 7g are logically equivalent. < 


Show that (p A gq) > (p V q) is a tautology. 


Solution: To show that this statement is a tautology, we will use logical equivalences to demon- 
strate that it is logically equivalent to T. (Note: This could also be done using a truth table.) 


(Pq) > (PVG) =-(PAG)V (PV 4) by Example 3 
=(4-pV-7q)V (pV q) | bythe first De Morgan law 


=(-pVv p)V (-qVq) | by the associative and commutative 
laws for disjunction 

=TvT by Example 1 and the commutative 
law for disjunction 


=T by the domination law 


Propositional Satisfiability 


A compound proposition is satisfiable if there is an assignment of truth values to its variables that 
makes it true. When no such assignments exists, that is, when the compound proposition is false 
for all assignments of truth values to its variables, the compound proposition is unsatisfiable. 
Note that a compound proposition is unsatisfiable if and only if its negation is true for all 
assignments of truth values to the variables, that is, if and only if its negation is a tautology. 
When we find a particular assignment of truth values that makes a compound proposition 
true, we have shown that it is satisfiable; such an assignment is called a solution of this particular 
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satisfiability problem. However, to show that a compound proposition is unsatisfiable, we need 
to show that every assignment of truth values to its variables makes it false. Although we can 
always use a truth table to determine whether a compound proposition is satisfiable, it is often 
more efficient not to, as Example 9 demonstrates. 


EXAMPLE9 Determine whether each of the compound propositions (pV —-q) A (gq V-7r)A 
(rFV—ap) (pVqVr)A(7pvV7qvr-r), and (pV7q)A(qV ar) AV ap) A 
(pVqVr)A(A=pVv 7q V =r) is satisfiable. 


Solution: Instead of using truth table to solve this problem, we will reason about truth values. 
Note that (p V —=qg) A (¢ V =r) A (r V =p) is true when the three variable p, g, and r have 
the same truth value (see Exercise 40 of Section 1.1). Hence, it is satisfiable as there is at 
least one assignment of truth values for p, q, and r that makes it true. Similarly, note that 
(pV qVr)A (=p Vv 7q V =r) is true when at least one of p, g, and r is true and at least one 
is false (see Exercise 41 of Section 1.1). Hence, (p V g Vr) A (=p V -q V =r) is satisfiable, 
as there is at least one assignment of truth values for p, g, andr that makes it true. 

Finally, note that for (p V -q) A (qV ~r)A(rVap)A(pVqgVr)A (Apv-7qgv-7r) 
to be true, (p V -q) A (qV mr) A(r V ap) and (pV g Vr) A (Ap V 7q V =r) must both 
be true. For the first to be true, the three variables must have the same truth values, and 
for the second to be true, at least one of three variables must be true and at least one must 
be false. However, these conditions are contradictory. From these observations we conclude 
that no assignment of truth values to p, g, and r makes (p V =qg) A (qV =r) A(r V ap) A 
(pV qVr)A (=pVv 7q V =r) true. Hence, it is unsatisfiable. 


AUGUSTA ADA, COUNTESS OF LOVELACE (1815-1852) Augusta Ada was the only child from the 
marriage of the famous poet Lord Byron and Lady Byron, Annabella Millbanke, who separated when Ada 
was | month old, because of Lord Byron’s scandalous affair with his half sister. The Lord Byron had quite a 
reputation, being described by one of his lovers as “mad, bad, and dangerous to know.” Lady Byron was noted for 
her intellect and had a passion for mathematics; she was called by Lord Byron “The Princess of Parallelograms.” 
Augusta was raised by her mother, who encouraged her intellectual talents especially in music and mathematics, 
to counter what Lady Byron considered dangerous poetic tendencies. At this time, women were not allowed to 
attend universities and could not join learned societies. Nevertheless, Augusta pursued her mathematical studies 
independently and with mathematicians, including William Frend. She was also encouraged by another female 
mathematician, Mary Somerville, and in 1834 at a dinner party hosted by Mary Somerville, she learned about Charles Babbage’s 
ideas for a calculating machine, called the Analytic Engine. In 1838 Augusta Ada married Lord King, later elevated to Earl of 
Lovelace. Together they had three children. 

Augusta Ada continued her mathematical studies after her marriage. Charles Babbage had continued work on his Analytic 
Engine and lectured on this in Europe. In 1842 Babbage asked Augusta Ada to translate an article in French describing Babbage’s 
invention. When Babbage saw her translation, he suggested she add her own notes, and the resulting work was three times the 
length of the original. The most complete accounts of the Analytic Engine are found in Augusta Ada’s notes. In her notes, she 
compared the working of the Analytic Engine to that of the Jacquard loom, with Babbage’s punch cards analogous to the cards used 
to create patterns on the loom. Furthermore, she recognized the promise of the machine as a general purpose computer much better 
than Babbage did. She stated that the “engine is the material expression of any indefinite function of any degree of generality and 
complexity.” Her notes on the Analytic Engine anticipate many future developments, including computer-generated music. Augusta 
Ada published her writings under her initials A.A.L. concealing her identity as a woman as did many women at a time when women 
were not considered to be the intellectual equals of men. After 1845 she and Babbage worked toward the development of a system 
to predict horse races. Unfortunately, their system did not work well, leaving Augusta Ada heavily in debt at the time of her death 
at an unfortunately young age from uterine cancer. 

In 1953 Augusta Ada’s notes on the Analytic Engine were republished more than 100 years after they were written, and after 
they had been long forgotten. In his work in the 1950s on the capacity of computers to think (and his famous Turing Test), Alan 
Turing responded to Augusta Ada’s statement that “The Analytic Engine has no pretensions whatever to originate anything. It can do 
whatever we know how to order it to perform.” This “dialogue” between Turing and Augusta Ada is still the subject of controversy. 
Because of her fundamental contributions to computing, the programming language Ada is named in honor of the Countess of 
Lovelace. 
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Links 


FIGURE 1 A 9x 9 Sudoku puzzle. 


Applications of Satisfiability 


Many problems, in diverse areas such as robotics, software testing, computer-aided design, 
machine vision, integrated circuit design, computer networking, and genetics, can be modeled 
in terms of propositional satisfiability. Although most of these applications are beyond the 
scope of this book, we will study one application here. In particular, we will show how to use 
propositional satisfiability to model Sudoku puzzles. 


SUDOKU A Sudoku puzzle is represented by a 9 x 9 grid made up of nine 3 x 3 subgrids, 
known as blocks, as shown in Figure |. For each puzzle, some of the 81 cells, called givens, 
are assigned one of the numbers 1, 2, ..., 9, and the other cells are blank. The puzzle is solved 
by assigning a number to each blank cell so that every row, every column, and every one of the 
nine 3 x 3 blocks contains each of the nine possible numbers. Note that instead of using a9 x 9 
grid, Sudoku puzzles can be based on n* x n? grids, for any positive integer n, with the n? x n? 
grid made up of n? n x n subgrids. 


The popularity of 


The modern game of Sudoku was apparently designed 
in the late 1970s by an American puzzle designer. The basic ideas of Sudoku date back even 
further; puzzles printed in French newspapers in the 1890s were quite similar, but not identical, 
to modern Sudoku. 

Sudoku puzzles designed for entertainment have two additional important properties. First, 
they have exactly one solution. Second, they can be solved using reasoning alone, that is, without 
resorting to searching all possible assignments of numbers to the cells. As a Sudoku puzzle is 
solved, entries in blank cells are successively determined by already known values. For instance, 
in the grid in Figure 1, the number 4 must appear in exactly one cell in the second row. How 
can we determine which of the seven blank cells it must appear? First, we observe that 4 cannot 
appear in one of the first three cells or in one of the last three cells of this row, because it already 
appears in another cell in the block each of these cells is in. We can also see that 4 cannot appear 
in the fifth cell in this row, as it already appears in the fifth column in the fourth row. This means 
that 4 must appear in the sixth cell of the second row. 

Many strategies based on logic and mathematics have been devised for solving Sudoku 
puzzles (see [Dal0], for example). Here, we discuss one of the ways that have been developed 
for solving Sudoku puzzles with the aid of a computer, which depends on modeling the puzzle as 
a propositional satisfiability problem. Using the model we describe, particular Sudoku puzzles 
can be solved using software developed to solve satisfiability problems. Currently, Sudoku 
puzzles can be solved in less than 10 milliseconds this way. It should be noted that there are 
many other approaches for solving Sudoku puzzles via computers using other techniques. 


? 


It is tricky setting up the 
two inner indices so that 
all nine cells in each 
square block are 
examined. 
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To encode a Sudoku puzzle, let p(i, j, n) denote the proposition that is true when the number 
n is in the cell in the ith row and jth column. There are 9 x 9 x 9 = 729 such propositions, as 
i, j, and n all range from | to 9. For example, for the puzzle in Figure 1, the number 6 is given 
as the value in the fifth row and first column. Hence, we see that p(5, 1, 6) is true, but p(5, 7, 6) 
is false for 7 = 2,3,...,9. 

Given a particular Sudoku puzzle, we begin by encoding each of the given values. Then, 
we construct compound propositions that assert that every row contains every number, every 
column contains every number, every 3 x 3 block contains every number, and each cell contains 
no more than one number. It follows, as the reader should verify, that the Sudoku puzzle is solved 
by finding an assignment of truth values to the 729 propositions p(i, j,n) withi, 7, and each 
ranging from | to 9 that makes the conjunction of all these compound propositions true. After 
listing these assertions, we will explain how to construct the assertion that every row contains 
every integer from | to 9. We will leave the construction of the other assertions that every column 
contains every number and each of the nine 3 x 3 blocks contains every number to the exercises. 


@ For each cell with a given value, we assert p(i, j,n) when the cell in row 7 and column 
j has the given value n. 


@ We assert that every row contains every number: 


9 9 9 
AAV eein 


i=l n=1 j=1 
® We assert that every column contains every number: 


9 9 9 


NAV PGi 


j=ln=1i=1 


m@ We assert that each of the nine 3 x 3 blocks contains every number: 


2 2 O° 3 3 
AAAVN V PGr+i3s ti 


r=0 s=0n=1 i=1 j=1 


® To assert that no cell contains more than one number, we take the conjunction over all 
values of n,n’, i, and j where each variable ranges from | to9 andn £ n’ of p(i, j,n) > 


=p, j,i). 


We now explain how to construct the assertion that every row contains every number. 
First, to assert that row i contains the number n, we form Vee , PG, j,n). To assert that 
row i contains all n numbers, we form the conjunction of these disjunctions over all nine 
possible values of n, giving us Pe ak Ve pCi, j,n). Finally, to assert that every row contains 


every number, we take the conjunction of fa 4 Va pG, j,) over all nine rows. This gives 


us ian s 1 Via p(i, j, n). (Exercises 65 and 66 ask for explanations of the assertions that 
every column contains every number and that each of the nine 3 x 3 blocks contains every 
number.) 

Given a particular Sudoku puzzle, to solve this puzzle we can find a solution to the satisfia- 
bility problems that asks for a set of truth values for the 729 variables p(i, j,) that makes the 
conjunction of all the listed assertions true. 
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Solving Satisfiability Problems 


A truth table can be used to determine whether a compound proposition is satisfiable, or equiv- 
alently, whether its negation is a tautology (see Exercise 60). This can be done by hand for 
a compound proposition with a small number of variables, but when the number of variables 
grows, this becomes impractical. For instance, there are 220 — 1,048,576 rows in the truth ta- 
ble for a compound proposition with 20 variables. Clearly, you need a computer to help you 
determine, in this way, whether a compound proposition in 20 variables is satisfiable. 

When many applications are modeled, questions concerning the satisfiability of compound 
propositions with hundreds, thousands, or millions of variables arise. Note, for example, that 
when there are 1000 variables, checking every one of the 2!°°° (a number with more than 300 
decimal digits) possible combinations of truth values of the variables in a compound proposition 
cannot be done by a computer in even trillions of years. No procedure is known that a com- 
puter can follow to determine in a reasonable amount of time whether an arbitrary compound 
proposition in such a large number of variables is satisfiable. However, progress has been made 
developing methods for solving the satisfiability problem for the particular types of compound 

tinks Red propositions that arise in practical applications, such as for the solution of Sudoku puzzles. 
Many computer programs have been developed for solving satisfiability problems which have 
practical use. In our discussion of the subject of algorithms in Chapter 3, we will discuss this 
question further. In particular, we will explain the important role the propositional satisfiability 
problem plays in the study of the complexity of algorithms. 


Exercises 
1. Use truth tables to verify these equivalences. b) (DAQAr=PACADN). 
a) DAT=p b) pVF=p 5. Use a truth table to verify the distributive law 
c) pAF=F d) pvT=T PAGVr)=(PAqQV (PAP). 
e) pYVp=P f) pAp=pP 6. Use a truth table to verify the first De Morgan law 


2. Show that —(—>p) and p are logically equivalent. 


(pg) = pV 7g. 


- Use truth tables to verify the commutative laws 7. Use De Morgan’s laws to find the negation of each of the 
a) pVq=4qVp. b) pAG=q Ap. following statements. 

. Use truth tables to verify the associative laws a) Jan is rich and happy. 
a) (PVqQ)Vr=pV(qvr). b) Carlos will bicycle or run tomorrow. 


q HENRY MAURICE SHEFFER (1883-1964) Henry Maurice Sheffer, born to Jewish parents in the western 
| Ukraine, emigrated to the United States in 1892 with his parents and six siblings. He studied at the Boston Latin 
| School before entering Harvard, where he completed his undergraduate degree in 1905, his master’s in 1907, 
and his Ph.D. in philosophy in 1908. After holding a postdoctoral position at Harvard, Henry traveled to Europe 
on a fellowship. Upon returning to the United States, he became an academic nomad, spending one year each 
at the University of Washington, Cornell, the University of Minnesota, the University of Missouri, and City 
y College in New York. In 1916 he returned to Harvard as a faculty member in the philosophy department. He 
: a remained at Harvard until his retirement in 1952. 

. Sheffer introduced what is now known as the Sheffer stroke in 1913; it became well known only after its use 
in the 1925 edition of Whitehead and Russell’s Principia Mathematica. In this same edition Russell wrote that Sheffer had invented 
a powerful method that could be used to simplify the Principia. Because of this comment, Sheffer was something of a mystery man 
to logicians, especially because Sheffer, who published little in his career, never published the details of this method, only describing 
it in mimeographed notes and in a brief published abstract. 

Sheffer was a dedicated teacher of mathematical logic. He liked his classes to be small and did not like auditors. When strangers 
appeared in his classroom, Sheffer would order them to leave, even his colleagues or distinguished guests visiting Harvard. Sheffer 
was barely five feet tall; he was noted for his wit and vigor, as well as for his nervousness and irritability. Although widely liked, he 
was quite lonely. He is noted for a quip he spoke at his retirement: “Old professors never die, they just become emeriti.” Sheffer is 
also credited with coining the term “Boolean algebra” (the subject of Chapter 12 of this text). Sheffer was briefly married and lived 
most of his later life in small rooms at a hotel packed with his logic books and vast files of slips of paper he used to jot down his 
ideas. Unfortunately, Sheffer suffered from severe depression during the last two decades of his life. 
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LS 10. 


11. 


12. 


. Determine whether (—g A (p > q)) > 


c) Mei walks or takes the bus to class. 
d) Ibrahim is smart and hard working. 


. Use De Morgan’s laws to find the negation of each of the 


following statements. 


a) Kwame will take a job in industry or go to graduate 
school. 

b) Yoshiko knows Java and calculus. 

c) James is young and strong. 

d) Rita will move to Oregon or Washington. 


. Show that each of these conditional statements is a tau- 


tology by using truth tables. 
a) (pAq) > P 

ce) ~p> (p> @) d) (pPAq) > (p> @) 
e) "(p> gp f) -(p> q) > 74 
Show that each of these conditional statements is a tau- 
tology by using truth tables. 

a) [=pA(PpYg)l> 4 

b) [(p>egaAgqrrl> pr) 

°.) [pPA~P>gl>¢ 

qd) (pyvgAPrrnAgdarl>r 

Show that each conditional statement in Exercise 9 is a 
tautology without using truth tables. 

Show that each conditional statement in Exercise 10 is a 
tautology without using truth tables. 


b) p> (pv q) 


. Use truth tables to verify the absorption laws. 


a) pV(pAq)=Pp b) pA (pV q) =P 


. Determine whether (—p A (p > q)) > nq is a tautol- 


ogy. 


1p is a tautol- 
ogy. 


Each of Exercises 16-28 asks you to show that two compound 
propositions are logically equivalent. To do this, either show 
that both sides are true, or that both sides are false, for exactly 
the same combinations of truth values of the propositional 
variables in these expressions (whichever is easier). 


16. 
17. 
18. 
19. 
20. 
21. 
22. 
23. 
24. 
25. 
26. 
27. 


28. 


Show that p <= q and (p A q) V (~=p A 7q) are logically 
equivalent. 

Show that —(p <= q) and p = 77 are logically equiva- 
lent. 

Show that p — gq and-—g — —pare logically equivalent. 
Show that =p < q and p = 7g are logically equivalent. 
Show that —=(p ® q) and p = q are logically equivalent. 
Show that —(p <= q) and =p < gq are logically equiva- 
lent. 

Show that (p > q) A (p > r)and p > (g Ar)are log- 
ically equivalent. 

Show that (p > r) A (¢ > r) and (p V q) —> rare log- 
ically equivalent. 

Show that (p > q) V (p > r) and p > (g Vr) are log- 
ically equivalent. 

Show that (p > r) V (¢ > r)and(p Aq) > rare log- 
ically equivalent. 

Show that—p — (q > r)andg —> (p V r)are logically 
equivalent. 

Show that p < g and (p > qg) A (q — p) are logically 
equivalent. 

Show that p <= q and-=p < 7g are logically equivalent. 


29. 


32. 


33. 


” 30. 
31. 
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Show that (p > q) A (q > r) > (p > Fr) is a tautol- 
ogy. 

Show that (p V g) A (-=p Vr) > (¢ Vr) isa tautology. 
Show that (p > gq) > r and p > (q — r) are not log- 
ically equivalent. 

Show that (p Aq) > r and (p > r) A (q > r) are not 
logically equivalent. 

Show that (p>q)—>(r—>s) and (p>r)—> 
(q — s) are not logically equivalent. 


The dual of a compound proposition that contains only the 
logical operators V, A, and — is the compound proposition 
obtained by replacing each v by A, each A by v, each T 
by F, and each F by T. The dual of s is denoted by s*. 


34. 


35. 


#* 39, 


40. 


41. 


c= 42, 


Find the dual of each of these compound propositions. 
a) pV —7q b) pPA@GV(rAT)) 
ce) (PA7“qQ)V GAF) 

Find the dual of each of these compound propositions. 
a) pA-qA-r b) (pPAGAr)Vs 
c) (PVF) A@vT) 


. When does s* = s, where s is a compound proposition? 
. Show that (s*)* = s when s is a compound proposition. 
. Show that the logical equivalences in Table 6, except for 


the double negation law, come in pairs, where each pair 
contains compound propositions that are duals of each 
other. 


Why are the duals of two equivalent compound proposi- 
tions also equivalent, where these compound propositions 
contain only the operators A, V, and 7? 


Find a compound proposition involving the propositional 
variables p, g, and r that is true when p and gq are true 
and r is false, but is false otherwise. [Hint: Use a con- 
junction of each propositional variable or its negation. ] 


Find a compound proposition involving the propositional 
variables p, g, andr that is true when exactly two of p, q, 
and r are true and is false otherwise. [Hint: Form a dis- 
junction of conjunctions. Include a conjunction for each 
combination of values for which the compound proposi- 
tion is true. Each conjunction should include each of the 
three propositional variables or its negations. ] 


Suppose that a truth table in n propositional variables is 
specified. Show that a compound proposition with this 
truth table can be formed by taking the disjunction of 
conjunctions of the variables or their negations, with one 
conjunction included for each combination of values for 
which the compound proposition is true. The resulting 
compound proposition is said to be in disjunctive nor- 
mal form. 


A collection of logical operators is called functionally com- 
plete if every compound proposition is logically equivalent to 
a compound proposition involving only these logical opera- 


tors. 
43. 


Show that —, A, and v form a functionally complete col- 
lection of logical operators. [Hint: Use the fact that every 
compound proposition is logically equivalent to one in 
disjunctive normal form, as shown in Exercise 42.] 
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Show that — and A form a functionally complete col- 
lection of logical operators. [Hint: First use a De Mor- 
gan law to show that p V q is logically equivalent to 
(7p A749). 

Show that — and v form a functionally complete collec- 
tion of logical operators. 


The following exercises involve the logical operators NAND 
and NOR. The proposition p NAND gq is true when either p 
or q, or both, are false; and it is false when both p and q are 
true. The proposition p NOR q is true when both p and q are 
false, and it is false otherwise. The propositions p NAND q 
and p NOR q are denoted by p | q and p | q, respectively. 
(The operators | and | are called the Sheffer stroke and the 
Peirce arrow after H. M. Sheffer and C. S. Peirce, respec- 
tively.) 


46. 
47. 
48. 
49. 
50. 


*51. 
52. 
53. 
54, 

#55, 


56. 


57. 


Predicates and Quantifiers 


Construct a truth table for the logical operator NAND. 

Show that p | q is logically equivalent to =(p A q). 

Construct a truth table for the logical operator NOR. 

Show that p | g is logically equivalent to —(p Vv q). 

In this exercise we will show that {|} is a functionally 

complete collection of logical operators. 

a) Show that p | p is logically equivalent to sp. 

b) Show that (p | g) J (p | @) is logically equivalent 
topVvq. 

c) Conclude from parts (a) and (b), and Exercise 49, that 
{)} is a functionally complete collection of logical 
operators. 


Find a compound proposition logically equivalent to 
P — q using only the logical operator |. 

Show that {|} is a functionally complete collection of log- 
ical operators. 

Show that p | qg and q | p are equivalent. 

Show that p | (¢ |r) and (p | g) | r are not equivalent, 
so that the logical operator | is not associative. 

How many different truth tables of compound proposi- 
tions are there that involve the propositional variables p 
and q? 

Show that if p, g, andr are compound propositions such 
that p and q are logically equivalent and q and r are log- 
ically equivalent, then p and r are logically equivalent. 
The following sentence is taken from the specification of 
a telephone system: “If the directory database is opened, 
then the monitor is put in a closed state, if the system is 
not in its initial state.” This specification is hard to under- 
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59. 


60. 


61. 


62. 


63. 


64. 


65. 


* 66. 


stand because it involves two conditional statements. Find 
an equivalent, easier-to-understand specification that in- 
volves disjunctions and negations but not conditional 
statements. 


How many of the disjunctions p V -g, ~=pV q,qvVr, 
q V —r, and —q V —r can be made simultaneously true 
by an assignment of truth values to p, g, andr? 


How many of the disjunctions pV—-qVs, ap V 
rVS, ApV-7rv-7s, 7ApVqgVrs, qVrvr7s, 
qV-rV78,4pV7qV75,pVrvs,andpVvrv-7s 


can be made simultaneously true by an assignment of 
truth values to p,g,r, ands? 


Show that the negation of an unsatisfiable compound 
proposition is a tautology and the negation of acompound 
proposition that is a tautology is unsatisfiable. 


Determine whether each of these compound propositions 

is satisfiable. 

a) (pV 7g) \ (7p Vv gq) A (PY 79) 

b) Pe QAD> 79) ACP > g)A (CP > 79) 

e) (pogAcps® 

Determine whether each of these compound propositions 

is satisfiable. 

a) (DVqGV-r) A (pV 7q V 75) A (pV ary as) A 
(7p V -qV 75) A (DV qv 7s) 

b) OPV -aVT)ACPYV GV 75) A (PV 7qV 

S)A(ApVaArVv=as)A(pVqVr-r) A (pv 

=r V 7s) 

ce) (PV GVA (pV 7qV>s)A (GV 7rVvs) A 
(ApVrVvs)A(pVgVr7s)A (pV 7qV 7r) A 
(=pV7qVs)A (A=pVv-rv-s) 

Show how the solution of a given 4 x 4 Sudoku puzzle 

can be found by solving a satisfiability problem. 


Construct a compound proposition that asserts that ev- 
ery cell of a 9 x 9 Sudoku puzzle contains at least one 
number. 


Explain the steps in the construction of the compound 
proposition given in the text that asserts that every col- 
umn of a9 x 9 Sudoku puzzle contains every number. 


Explain the steps in the construction of the compound 
proposition given in the text that asserts that each of the 
nine 3 x 3 blocks of a9 x 9 Sudoku puzzle contains ev- 
ery number. 


Introduction 


Propositional logic, studied in Sections 1.1—1.3, cannot adequately express the meaning of all 
statements in mathematics and in natural language. For example, suppose that we know that 


“Every computer connected to the university network is functioning properly.” 


EXAMPLE 1 
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No rules of propositional logic allow us to conclude the truth of the statement 
“MATHS is functioning properly,” 


where MATH3 is one of the computers connected to the university network. Likewise, we cannot 
use the rules of propositional logic to conclude from the statement 


“CS? is under attack by an intruder,” 
where CS2 is a computer on the university network, to conclude the truth of 
“There is a computer on the university network that is under attack by an intruder.” 


In this section we will introduce a more powerful type of logic called predicate logic. We 
will see how predicate logic can be used to express the meaning of a wide range of statements 
in mathematics and computer science in ways that permit us to reason and explore relationships 
between objects. To understand predicate logic, we first need to introduce the concept of a 
predicate. Afterward, we will introduce the notion of quantifiers, which enable us to reason with 
statements that assert that a certain property holds for all objects of a certain type and with 
statements that assert the existence of an object with a particular property. 


Predicates 
Statements involving variables, such as 
x>3” “x=yt3P “xty=z,” 
and 

“computer x is under attack by an intruder,” 
and 

“computer x is functioning properly,” 


are often found in mathematical assertions, in computer programs, and in system specifications. 
These statements are neither true nor false when the values of the variables are not specified. In 
this section, we will discuss the ways that propositions can be produced from such statements. 

The statement “x is greater than 3” has two parts. The first part, the variable x, is the subject 
of the statement. The second part—the predicate, “is greater than 3’”—refers to a property that 
the subject of the statement can have. We can denote the statement “x is greater than 3” by P(x), 
where P denotes the predicate “is greater than 3” and x is the variable. The statement P(x) is 
also said to be the value of the propositional function P at x. Once a value has been assigned 
to the variable x, the statement P(x) becomes a proposition and has a truth value. Consider 
Examples | and 2. 


Let P(x) denote the statement “x > 3.” What are the truth values of P(4) and P(2)? 


Solution: We obtain the statement P(4) by setting x = 4 in the statement “x > 3.’ Hence, 
P(4), which is the statement “4 > 3,” is true. However, P(2), which is the statement “2 > 3,” 
is false. < 


38 


1 / The Foundations: Logic and Proofs 


EXAMPLE 2 Let A(x) denote the statement “Computer x is under attack by an intruder.” Suppose that of the 
computers on campus, only CS2 and MATH are currently under attack by intruders. What are 
truth values of A(CS1), A(CS2), and A(MATH1)? 


Solution: We obtain the statement A(CS1) by setting x = CS1 in the statement “Computer x 
is under attack by an intruder.” Because CS1 is not on the list of computers currently under 
attack, we conclude that A(CS1) is false. Similarly, because CS2 and MATHI are on the list of 
computers under attack, we know that A(CS2) and A(MATH1) are true. < 


We can also have statements that involve more than one variable. For instance, consider the 
statement “x = y + 3.” We can denote this statement by Q(x, y), where x and y are variables 
and Q is the predicate. When values are assigned to the variables x and y, the statement Q(x, y) 
has a truth value. 


EXAMPLE3 Let Q(x, y) denote the statement “x = y + 3.” What are the truth values of the propositions 
Q(1, 2) and Q(3, 0)? 


Extra ~ Solution: To obtain Q(1, 2), set x = | and y = 2 in the statement Q(x, y). Hence, Q(1, 2) is 
Examples the statement “1 = 2 + 3,” which is false. The statement Q(3, 0) is the proposition “3 = 0 + 3,” 
which is true. < 


Links 


CHARLES SANDERS PEIRCE (1839-1914) Many consider Charles Peirce, born in Cambridge, Mas- 
sachusetts, to be the most original and versatile American intellect. He made important contributions to an 
amazing number of disciplines, including mathematics, astronomy, chemistry, geodesy, metrology, engineer- 
ing, psychology, philology, the history of science, and economics. Peirce was also an inventor, a lifelong student 
of medicine, a book reviewer, a dramatist and an actor, a short story writer, a phenomenologist, a logician, and a 
metaphysician. He is noted as the preeminent system-building philosopher competent and productive in logic, 
mathematics, and a wide range of sciences. He was encouraged by his father, Benjamin Peirce, a professor of 
mathematics and natural philosophy at Harvard, to pursue a career in science. Instead, he decided to study logic 
and scientific methodology. Peirce attended Harvard (1855-1859) and received a Harvard master of arts degree 
(1862) and an advanced degree in chemistry from the Lawrence Scientific School (1863). 

In 1861, Peirce became an aide in the U.S. Coast Survey, with the goal of better understanding scientific methodology. His service 
for the Survey exempted him from military service during the Civil War. While working for the Survey, Peirce did astronomical and 
geodesic work. He made fundamental contributions to the design of pendulums and to map projections, applying new mathematical 
developments in the theory of elliptic functions. He was the first person to use the wavelength of light as a unit of measurement. 
Peirce rose to the position of Assistant for the Survey, a position he held until forced to resign in 1891 when he disagreed with the 
direction taken by the Survey’s new administration. 

While making his living from work in the physical sciences, Peirce developed a hierarchy of sciences, with mathematics at the 
top rung, in which the methods of one science could be adapted for use by those sciences under it in the hierarchy. During this time, 
he also founded the American philosophical theory of pragmatism. 

The only academic position Peirce ever held was lecturer in logic at Johns Hopkins University in Baltimore (1879-1884). His 
mathematical work during this time included contributions to logic, set theory, abstract algebra, and the philosophy of mathematics. 
His work is still relevant today, with recent applications of this work on logic to artificial intelligence. Peirce believed that the study 
of mathematics could develop the mind’s powers of imagination, abstraction, and generalization. His diverse activities after retiring 
from the Survey included writing for periodicals, contributing to scholarly dictionaries, translating scientific papers, guest lecturing, 
and textbook writing. Unfortunately, his income from these pursuits was insufficient to protect him and his second wife from abject 
poverty. He was supported in his later years by a fund created by his many admirers and administered by the philosopher William 
James, his lifelong friend. Although Peirce wrote and published voluminously in a vast range of subjects, he left more than 100,000 
pages of unpublished manuscripts. Because of the difficulty of studying his unpublished writings, scholars have only recently started 
to understand some of his varied contributions. A group of people is devoted to making his work available over the Internet to bring 
a better appreciation of Peirce’s accomplishments to the world. 


EXAMPLE 4 


EXAMPLE 5 


EXAMPLE 6 


EXAMPLE 7 
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Let A(c, n) denote the statement “Computer c is connected to network n,” where c is a variable 
representing a computer and n is a variable representing a network. Suppose that the computer 
MATH is connected to network CAMPUS2, but not to network CAMPUS1. What are the 
values of A(C€MATHI, CAMPUS1) and A(MATHI, CAMPUS2)? 


Solution: Because MATH is not connected to the CAMPUS 1 network, we see that ACMATH1, 
CAMPUS1) is false. However, because MATH is connected to the CAMPUS2 network, we 
see that AAMATH1, CAMPUS2) is true. < 


Similarly, we can let R(x, y, z) denote the statement ‘x + y = z.” When values are assigned 
to the variables x, y, and z, this statement has a truth value. 


What are the truth values of the propositions R(1, 2, 3) and R(O, 0, 1)? 


Solution: The proposition R(1, 2,3) is obtained by setting x = 1, y = 2, and z = 3 in the 
statement R(x, y, z). We see that R(1, 2, 3) is the statement “1 + 2 = 3,” which is true. Also 
note that R(O, 0, 1), which is the statement “O + 0 = 1,” is false. < 


In general, a statement involving the n variables x1, x2, ..., X, can be denoted by 
P(x1,X2,...,Xn). 


A statement of the form P(x, x2,...,X,) 1s the value of the propositional function P at the 
n-tuple (x1, X2,...,Xn), and P is also called an n-place predicate or a n-ary predicate. 
Propositional functions occur in computer programs, as Example 6 demonstrates. 


Consider the statement 
if x > Othenx :=x+1. 


When this statement is encountered in a program, the value of the variable x at that point in the 
execution of the program is inserted into P(x), which is “x > 0.’ If P(x) is true for this value 
of x, the assignment statement x := x + | is executed, so the value of x is increased by 1. If 
P(x) is false for this value of x, the assignment statement is not executed, so the value of x is 
not changed. 


PRECONDITIONS AND POSTCONDITIONS Predicates are also used to establish the 
correctness of computer programs, that is, to show that computer programs always produce the 
desired output when given valid input. (Note that unless the correctness of a computer program 
is established, no amount of testing can show that it produces the desired output for all input 
values, unless every input value is tested.) The 


s Example 7 illustrates, we use predicates to describe both 
preconditions and postconditions. We will study this process in greater detail in Section 5.5. 


Consider the following program, designed to interchange the values of two variables x and y. 


temp := xX 
KX i= y 
y := temp 


Find predicates that we can use as the precondition and the postcondition to verify the correctness 
of this program. Then explain how to use them to verify that for all valid input the program does 
what is intended. 
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Assessment 


Assessment 


Solution: For the precondition, we need to express that x and y have particular values before 
we run the program. So, for this precondition we can use the predicate P(x, y), where P(x, y) 
is the statement “x = a and y = Db,” where a and D are the values of x and y before we run the 
program. Because we want to verify that the program swaps the values of x and y for all input 
values, for the postcondition we can use Q(x, y), where Q(x, y) is the statement “x = b and 
y=a,” 

To verify that the program always does what it is supposed to do, suppose that the precon- 
dition P(x, y) holds. That is, we suppose that the statement “x = a and y = b” is true. This 
means that x = a and y = D. The first step of the program, temp := x, assigns the value of x to 
the variable temp, so after this step we know that x = a, temp = a, and y = b. After the second 
step of the program, x := y, we know that x = b, temp = a, and y = D. Finally, after the third 
step, we know that x = b, temp = a, and y = a. Consequently, after this program is run, the 
postcondition Q(x, y) holds, that is, the statement “x = b and y = a” is true. < 


Quantifiers 


When the variables in a propositional function are assigned values, the resulting statement 
becomes a proposition with a certain truth value. However, there is another important way, called 
quantification, to create a proposition from a propositional function. 

In English, the words all, some, 
many, none, and few are used in quantifications. We will focus on two types of quantification 
here: universal quantification, which tells us that a predicate is true for every element under 
consideration, and existential quantification, which tells us that there is one or more element 
under consideration for which the predicate is true. The area of logic that deals with predicates 
and quantifiers is called the predicate calculus. 


THE UNIVERSAL QUANTIFIER Many mathematical statements assert that a property is 
true for all values of a variable in a particular domain, called the domain of discourse (or 
the universe of discourse), often just referred to as the domain. Such a statement is expressed 
using universal quantification. The universal quantification of P(x) for a particular domain is the 
proposition that asserts that(P(@)is tue for all values of x in this domain) Note that the domain 
specifies the possible values of the variable x. The meaning of the universal quantification 
of P(x) changes when we change the domain. The domain must always be specified when a 
universal quantifier is used; without it, the universal quantification of a statement is not defined. 


The universal quantification of P(x) 1s the statement 
“P(x) for all values of x in the domain.” 
The notation Vx P(x) denotes the universal quantification of P(x). Here V is called the 


universal quantifier. We read Vx P(x) as “for all x P(x)” or “for every x P(x). An element 
for which P(x) is false is called a counterexample of Vx P(x). 


The meaning of the universal quantifier is summarized in the first row of Table 1. We 
illustrate the use of the universal quantifier in Examples 8-13. 


EXAMPLE 8 


Extra p> 
Examples 2 


Remember that the truth 
value of Vx P(x) depends 
on the domain! 


EXAMPLE 9 


EXAMPLE 10 
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TABLE 1 Quantifiers. 


Vx P(x) P(x) is true for every x. There is an x for which P(x) is false. 
dx P(x) There is an x for which P(x) is true. P(x) is false for every x. 


Let P(x) be the statement “x + 1 > x.” What is the truth value of the quantification Vx P(x), 
where the domain consists of all real numbers? 


Solution: Because P(x) is true for all real numbers x, the quantification 
Vx P(x) 
is true. < 


Remark: Generally, an implicit assumption is made that all domains of discourse for quantifiers 
are nonempty. Note that if the domain is empty, then Vx P(x) is true for any propositional 
function P(x) because there are no elements x in the domain for which P(x) is false. 


Besides “for all” and “for every,” universal quantification can be expressed in many other 
ways, including “all of,’ “for each,” “given any,” “for arbitrary,” “for each,’ and “for any.” 


Remark: It is best to avoid using “for any x” because it is often ambiguous as to whether “any” 
means “every” or “some.” In some cases, “any” is unambiguous, such as when it is used in 
negatives, for example, “there is not any reason to avoid studying.” 


A statement Vx P (x) is false, where P(x) is a propositional function, if and only if P (x) is not 
always true when x is in the domain. One way to show that P(x) is not always true when x is in the 
domain is to find a counterexample to the statement Vx P (x). Note that a single counterexample is 
all we need to establish that Vx P (x) is false. Example 9 illustrates how counterexamples are used. 


Let Q(x) be the statement “x < 2.” What is the truth value of the quantification Vx Q(x), where 
the domain consists of all real numbers? 


Solution: Q(x) 1s not true for every real number x, because, for instance, Q(3) is false. That is, 
x = 3 is acounterexample for the statement Vx Q(x). Thus 


Vx O(x) 
is false. < 
Suppose that P(x) is “x* > 0.” To show that the statement Vx P(x) is false where the uni- 


verse of discourse consists of all integers, we give a counterexample. We see that x = 0 is a 
counterexample because x* = 0 when x = 0, so that x? is not greater than0 whenx =0. 4 


Looking for counterexamples to universally quantified statements is an important activity 
in the study of mathematics, as we will see in subsequent sections of this book. 

When all the elements in the domain can be listed—say, x1, x2, .. ., X,—it follows that the 
universal quantification Vx P(x) is the same as the conjunction 


P(x1) A P(x2) A+++ A PQn), 


because this conjunction 1s true if and only if P(x;), P(x2),..., P(x) are all true. 
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EXAMPLE 11 


EXAMPLE 12 


EXAMPLE 13 


What is the truth value of Vx P(x), where P(x) is the statement “x < 10” and the domain 
consists of the positive integers not exceeding 4? 


Solution: The statement Vx P(x) is the same as the conjunction 
PO) A PQO)A P(3)A P(4), 


because the domain consists of the integers 1, 2, 3, and 4. Because P(4), which is the statement 
“4? < 10,” is false, it follows that Vx P(x) is false. < 


What does the statement Vx N(x) mean if N(x) is “Computer x is connected to the network” 
and the domain consists of all computers on campus? 


Solution: The statement Vx N(x) means that for every computer x on campus, that computer x 
is connected to the network. This statement can be expressed in English as “Every computer on 
campus is connected to the network.” 


As we have pointed out, specifying the domain is mandatory when quantifiers are used. The 
truth value of a quantified statement often depends on which elements are in this domain, as 
Example 13 shows. 

What is the truth value of Vx(x? > x) if the domain consists of all real numbers? What is the 
truth value of this statement if the domain consists of all integers? 


Solution: The universal quantification Vx (x2 > x), where the domain consists of all real num- 
bers, is false. For example, (5)? Zz 5. Note that x? > x if and only if x2—x=x(x—1)>0. 
Consequently, x2 > x if and only if x < 0 or x > 1. It follows that Vx (x2 > x) is false if the 
domain consists of all real numbers (because the inequality is false for all real numbers x with 
0 < x < 1). However, if the domain consists of the integers, Vx (x2 > x) is true, because there 
are no integers x withO < x < 1. < 


THE EXISTENTIAL QUANTIFIER Many mathematical statements assert that there is an 
element with a certain property. Such statements are expressed using existential quantification. 
With existential quantification, we form a proposition that is true if and only if P(x) is true for 
at least one value of x in the domain. 


The existential quantification of P(x) is the proposition 
“There exists an element x in the domain such that P(x).” 


We use the notation 4x P(x) for the existential quantification of P(x). Here J is called the 
existential quantifier. 


A domain must always be specified when a statement 4x P(x) is used. Furthermore, the 
meaning of 4x P(x) changes when the domain changes. Without specifying the domain, the 
statement 4x P(x) has no meaning. 

Besides the phrase “there exists,” we can also express existential quantification in many other 
ways, such as by using the words “for some,” “for at least one,” or “there is.” The existential 
quantification 4x P(x) is read as 


“There is an x such that P(x),” 
“There is at least one x such that P(x),” 


or 


“For some x P(x).” 


EXAMPLE 14 


Extra p> 
Examples 


EXAMPLE 15 


Remember that the truth 
value of 4x P(x) depends 
on the domain! 


EXAMPLE 16 
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The meaning of the existential quantifier is summarized in the second row of Table 1. We 
illustrate the use of the existential quantifier in Examples 14-16. 


Let P(x) denote the statement “x > 3.’ What is the truth value of the quantification 4x P(x), 
where the domain consists of all real numbers? 


Solution: Because “x > 3” is sometimes true—for instance, when x = 4—the existential quan- 
tification of P(x), which is 4x P(x), is true. < 


Observe that the statement 4x P(x) is false if and only if there is no element x in the domain 
for which P(x) is true. That is, 4x P(x) is false if and only if P(x) is false for every element of 
the domain. We illustrate this observation in Example 15. 


Let Q(x) denote the statement “x = x + 1.’ What is the truth value of the quantification 4x Q(x), 
where the domain consists of all real numbers? 


Solution: Because Q(x) is false for every real number x, the existential quantification of Q(x), 
which is 4x Q(x), is false. 


Remark: Generally, an implicit assumption is made that all domains of discourse for quantifiers 
are nonempty. If the domain is empty, then 4x Q(x) is false whenever Q(x) is a propositional 
function because when the domain is empty, there can be no element x in the domain for which 
Q(x) is true. 


When all elements in the domain can be listed—say, x1, x2, ..., X,— the existential quan- 
tification 4x P (x) is the same as the disjunction 


P(x1) V Pa) V-++ V PQ), 
because this disjunction is true if and only if at least one of P(x;), P(x2),..., P(%,) is true. 


What is the truth value of 3x P(x), where P(x) is the statement “x* > 10” and the universe of 
discourse consists of the positive integers not exceeding 4? 


Solution: Because the domain is {1, 2, 3, 4}, the proposition 4x P(x) is the same as the disjunc- 
tion 


PQ) Vv PQ)V P@G)V P(4). 
Because P(4), which is the statement “4? > 10,” is true, it follows that 4x P(x) is true. < 


It is sometimes helpful to think in terms of looping and searching when determining the 
truth value of a quantification. Suppose that there are n objects in the domain for the variable x. 
To determine whether Vx P(x) is true, we can loop through all 1 values of x to see whether 
P(x) is always true. If we encounter a value x for which P(x) is false, then we have shown that 
Vx P(x) is false. Otherwise, Vx P(x) is true. To see whether 4x P(x) is true, we loop through 
the n values of x searching for a value for which P(x) is true. If we find one, then 4x P(x) is 
true. If we never find such an x, then we have determined that 4x P(x) is false. (Note that this 
searching procedure does not apply if there are infinitely many values in the domain. However, 
it is still a useful way of thinking about the truth values of quantifications.) 
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We have now introduced universal and existential quan- 
tifiers. These are the most important quantifiers in mathematics and computer science. However, 
there is no limitation on the number of different quantifiers we can define, such as “there are 
exactly two,” “there are no more than three,” “there are at least 100,” and so on. Of these other 
quantifiers, the one that is most often seen is th : 
The notation 4!x P(x) [or 4;x P(x)] states “The iM 
(Other phrases for uniqueness quantification include “there is exactly one” and “there is one and 
only one.”) For instance, 4!x(x — 1 = 0), where the domain is the set of real numbers, states 
that there is a unique real number x such that x — | = 0. This is a true statement, as x = | is the 
unique real number such that x — 1 = 0. Observe that we can use quantifiers and propositional 
logic to express uniqueness (see Exercise 52 in Section 1.5), so the uniqueness quantifier can 
be avoided. Generally, it is best to stick with existential and universal quantifiers so that rules 
of inference for these quantifiers can be used. 


Quantifiers with Restricted Domains 


An abbreviated notation is often used to restrict the domain of a quantifier. In this nota- 
tion, a condition a variable must satisfy is included after the quantifier. This is illustrated in 
Example 17. We will also describe other forms of this notation involving set membership in 
Section 2.1. 


What do the statements Vx < 0 (x? > 0), Vy £0 (y3 # 0), and Az > 0 (z? = 2) mean, where 
the domain in each case consists of the real numbers? 


Solution: The statement Vx < 0 (x? > 0) states that for every real number x withx < 0, x7 >0. 
That is, it states “The square of a negative real number is positive.” This statement is the same 
as Wx(x <0 —> x2 >0). 

The statement Vy 4 0(y? 4 0) states that for every real number y with y ¢ 0, we have 
y? £0. That is, it states “The cube of every nonzero real number is nonzero.” Note that this 
statement is equivalent to Vy(y #0 > y? £0). 

Finally, the statement 4z > 0 (z2 = 2) states that there exists a real number z with z > 0 
such that z? = 2. That is, it states “There is a positive square root of 2.” This statement is 
equivalent to dz(z > 0 A z* = 2). < 


Note that the restriction of a universal quantification is the same as the universal quantifi- 
cation of a conditional statement. For instance, Vx < 0 (x? > 0) is another way of expressing 
Yx(x <0— x? > 0). On the other hand, the restriction of an existential quantification is the 
same as the existential quantification of a conjunction. For instance, Jz > 0 (z? = 2) is another 
way of expressing 4z(z > 0 A 2? = 2). 


Precedence of Quantifiers 


The quantifiers V and 3 have higher precedence than all logical operators from propositional 
calculus. For example, Vx P(x) V Q(x) is the disjunction of Vx P(x) and Q(x). In other words, 
it means (Vx P(x)) V Q(x) rather than Vx(P(x) V Q(x)). 


Binding Variables 


When a quantifier is used on the variable x, we say that this occurrence of the variable is bound. 
An occurrence of a variable that is not bound by a quantifier or set equal to a particular value 
is said to be free. All the variables that occur in a propositional function must be bound or set 
equal to a particular value to turn it into a proposition. This can be done using a combination of 
universal quantifiers, existential quantifiers, and value assignments. 


EXAMPLE 18 


DEFINITION 3 
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The part of a logical expression to which a quantifier is applied is called the scope of this 
quantifier. Consequently, a variable is free if it is outside the scope of all quantifiers in the 
formula that specify this variable. 


In the statement 4x(x + y = 1), the variable x is bound by the existential quantification 4x, but 
the variable y is free because it is not bound by a quantifier and no value is assigned to this 
variable. This illustrates that in the statement 4x (x + y = 1), x is bound, but y is free. 

In the statement 4x (P(x) A Q(x)) V Vx R(x), all variables are bound. The scope of the first 
quantifier, 4x, is the expression P(x) A Q(x) because 4x is applied only to P(x) A Q(x), and 
not to the rest of the statement. Similarly, the scope of the second quantifier, Vx, is the expression 
R(x). That is, the existential quantifier binds the variable x in P(x) A Q(x) and the universal 
quantifier Vx binds the variable x in R(x). Observe that we could have written our statement 
using two different variables x and y, as dx(P(x) A Q(x)) V VyR(y), because the scopes of 
the two quantifiers do not overlap. The reader should be aware that in common usage, the same 
letter is often used to represent variables bound by different quantifiers with scopes that do not 
overlap. 


Logical Equivalences Involving Quantifiers 


In Section 1.3 we introduced the notion of logical equivalences of compound propositions. We 
can extend this notion to expressions involving predicates and quantifiers. 


Statements involving predicates and quantifiers are logically equivalent if and only if they 
have the same truth value no matter which predicates are substituted into these statements 
and which domain of discourse is used for the variables in these propositional functions. 
We use the notation S = T to indicate that two statements S and T involving predicates and 
quantifiers are logically equivalent. 


Example 19 illustrates how to show that two statements involving predicates and quantifiers 
are logically equivalent. 


Show that Vx(P(x) A Q(x)) and Vx P(x) A Vx Q(x) are logically equivalent (where the same 
domain is used throughout). This logical equivalence shows that we can distribute a universal 
quantifier over a conjunction. Furthermore, we can also distribute an existential quantifier over 
a disjunction. However, we cannot distribute a universal quantifier over a disjunction, nor can 
we distribute an existential quantifier over a conjunction. (See Exercises 50 and 51.) 


Solution: To show that these statements are logically equivalent, we must show that they always 
take the same truth value, no matter what the predicates P and Q are, and no matter which 
domain of discourse is used. Suppose we have particular predicates P and Q, with a common 
domain. We can show that Vx(P(x) A Q(x)) and Vx P(x) A Vx Q(x) are logically equivalent 
by doing two things. First, we show that if Vx(P(x) A Q(x)) is true, then Vx P(x) A Vx Q(x) 
is true. Second, we show that if Vx P(x) A Vx Q(x) is true, then Vx(P(x) A Q(x)) is true. 

So, suppose that Vx(P(x) A Q(x)) is true. This means that if a is in the domain, then 
P(a) A Q(a) is true. Hence, P(a) is true and Q(a) is true. Because P(a) is true and Q(a) is 
true for every element in the domain, we can conclude that Vx P(x) and Vx Q(x) are both true. 
This means that Vx P(x) A Vx Q(x) is true. 

Next, suppose that Vx P(x) A Vx Q(x) is true. It follows that Vx P (x) is true and Vx Q(x) is 
true. Hence, if a is in the domain, then P(qa) is true and Q(a) is true [because P(x) and Q(x) 
are both true for all elements in the domain, there is no conflict using the same value of a here]. 
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Assessment 


It follows that for all a, P(a) A Q(a) is true. It follows that Vx( P(x) A Q(x)) is true. We can 
now conclude that 


Vx(P(x) A O(x)) = Vx P(x) A Vx Q(x). | 


Negating Quantified Expressions 


We will often want to consider the negation of a quantified expression. For instance, consider 
the negation of the statement 


“Every student in your class has taken a course in calculus.” 
This statement is a universal quantification, namely, 
Vx P(x), 


where P(x) is the statement “x has taken a course in calculus” and the domain consists of the 
students in your class. The negation of this statement is “It is not the case that every student in 
your class has taken a course in calculus.” This is equivalent to “There is a student in your class 
who has not taken a course in calculus.” And this is simply the existential quantification of the 
negation of the original propositional function, namely, 


dx =P (x). 
This example illustrates the following logical equivalence: 
SNelP(Ge) = ake =VPGe)), 


To show that =Vx P(x) and 4x P(x) are logically equivalent no matter what the propositional 
function P(x) is and what the domain is, first note that ~Vx P (x) is true if and only if Vx P(x) is 
false. Next, note that Vx P(x) is false if and only if there is an element x in the domain for which 
P(x) is false. This holds if and only if there is an element x in the domain for which —P (x) is 
true. Finally, note that there is an element x in the domain for which —P (x) is true if and only 
if Ix —P(x) is true. Putting these steps together, we can conclude that ~Vx P(x) is true if and 
only if 4x — P(x) is true. It follows that ~Vx P(x) and 4x —P (x) are logically equivalent. 

Suppose we wish to negate an existential quantification. For instance, consider the propo- 
sition “There is a student in this class who has taken a course in calculus.” This is the existential 
quantification 


dx Q(x), 


where Q(x) is the statement “x has taken a course in calculus.” The negation of this statement 
is the proposition “Tt is not the case that there is a student in this class who has taken a course in 
calculus.” This is equivalent to “Every student in this class has not taken calculus,” which is just 
the universal quantification of the negation of the original propositional function, or, phrased in 
the language of quantifiers, 


Vx =QO(x). 
This example illustrates the equivalence 
74x O(x) = Vx =QO(x). 


To show that —dx Q(x) and Vx —Q(x) are logically equivalent no matter what Q(x) is and what 
the domain is, first note that ~dx Q(x) is true if and only if 3x Q(x) is false. This is true if and 
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TABLE 2 De Morgan’s Laws for Quantifiers. 


Equivalent Statement When Is Negation True? When False? 


ax P(x) Vx-P(x) For every x, P(x) is false. There is an x for which 
P(x) is true. 
aAVx P(x) Ax-P (x) There is an x for which P(x) is true for every x. 
P(x) is false. 


only if no x exists in the domain for which Q(x) is true. Next, note that no x exists in the domain 
for which Q(x) is true if and only if Q(x) is false for every x in the domain. Finally, note that 
Q(x) is false for every x in the domain if and only if —Q(~) is true for all x in the domain, 
which holds if and only if Vx Q(x) is true. Putting these steps together, we see that —4x Q(x) 
is true if and only if Vx Q(x) is true. We conclude that ~dx Q(x) and Vx —Q(x) are logically 
equivalent. 

The rules for negations for quantifiers are called De Morgan’s laws for quantifiers. These 
rules are summarized in Table 2. 


Remark: When the domain of a predicate P(x) consists of n elements, where n is a positive 
integer greater than one, the rules for negating quantified statements are exactly the same as 
De Morgan’s laws discussed in Section 1.3. This is why these rules are called De Morgan’s 
laws for quantifiers. When the domain has n elements x1, x2, ..., Xn, it follows that ~Vx P(x) 
is the same as —(P(x1) A P(x2) A--+ A P(xn)), which is equivalent to —P(x,) V ~P(x2) V 
--» V —P(x,) by De Morgan’s laws, and this is the same as Jx—P(x). Similarly, -4x P(x) 
is the same as —(P(x1) V P(x2) V--+- V P(xn)), which by De Morgan’s laws is equivalent to 
aP(x1) \aP(x2) A--- A —P(x,), and this is the same as Vx P(x). 


We illustrate the negation of quantified statements in Examples 20 and 21. 


What are the negations of the statements “There is an honest politician” and “All Americans eat 
cheeseburgers”? 


Solution: Let H(x) denote ‘“‘x is honest.” Then the statement “There is an honest politician” 
is represented by 4xH (x), where the domain consists of all politicians. The negation of this 
statement is —AxH (x), which is equivalent to Vx -H (x). This negation can be expressed as 
“Every politician is dishonest.” (Note: In English, the statement “All politicians are not honest” 
is ambiguous. In common usage, this statement often means “Not all politicians are honest.” 
Consequently, we do not use this statement to express this negation.) 

Let C(x) denote “x eats cheeseburgers.” Then the statement “All Americans eat cheese- 
burgers” is represented by VxC (x), where the domain consists of all Americans. The negation 
of this statement is ~VxC (x), which is equivalent to Ix-C (x). This negation can be expressed 
in several different ways, including “Some American does not eat cheeseburgers” and “There 
is an American who does not eat cheeseburgers.” < 


What are the negations of the statements Vx (x2 > x) and x(x? = 2)? 


Solution: The negation of Vx (x? > x) is the statement ~Vx(x? > x), which is equivalent to 
dx7(x2 > x). This can be rewritten as 3x (x2 < x). The negation of 4x (x2 = 2) is the statement 
3x (x? = 2), which is equivalent to Yx-(x2 = 2). This can be rewritten as Wx (x2 4 2). The 
truth values of these statements depend on the domain. < 


We use De Morgan’s laws for quantifiers in Example 22. 
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EXAMPLE 22 


EXAMPLE 23 


Extra p> 
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Show that -Vx(P(x) > Q(x)) and 4x(P(x) A =Q(x)) are logically equivalent. 


Solution: By De Morgan’s law for universal quantifiers, we know that ~Vx(P(x) > Q(x)) 
and 4x(—=(P(x) + Q(x))) are logically equivalent. By the fifth logical equivalence in Table 7 
in Section 1.3, we know that —(P(x) > Q(x)) and P(x) A =Q(x) are logically equivalent 
for every x. Because we can substitute one logically equivalent expression for another in a 
logical equivalence, it follows that ~Vx(P(x) > Q(x)) and 4x(P(x) A =Q(x)) are logically 
equivalent. < 


Translating from English into Logical Expressions 


Translating sentences in English (or other natural languages) into logical expressions is a crucial 
task in mathematics, logic programming, artificial intelligence, software engineering, and many 
other disciplines. We began studying this topic in Section 1.1, where we used propositions to 
express sentences in logical expressions. In that discussion, we purposely avoided sentences 
whose translations required predicates and quantifiers. Translating from English to logical ex- 
pressions becomes even more complex when quantifiers are needed. Furthermore, there can 
be many ways to translate a particular sentence. (As a consequence, there is no “cookbook” 
approach that can be followed step by step.) We will use some examples to illustrate how to 
translate sentences from English into logical expressions. The goal in this translation is to pro- 
duce simple and useful logical expressions. In this section, we restrict ourselves to sentences 
that can be translated into logical expressions using a single quantifier; in the next section, we 
will look at more complicated sentences that require multiple quantifiers. 


Express the statement “Every student in this class has studied calculus” using predicates and 
quantifiers. 


Solution: First, we rewrite the statement so that we can clearly identify the appropriate quantifiers 
to use. Doing so, we obtain: 


“For every student in this class, that student has studied calculus.” 
Next, we introduce a variable x so that our statement becomes 
“For every student x in this class, x has studied calculus.” 


Continuing, we introduce C(x), which is the statement “x has studied calculus.” Consequently, 
if the domain for x consists of the students in the class, we can translate our statement as VxC (x). 

However, there are other correct approaches; different domains of discourse and other 
predicates can be used. The approach we select depends on the subsequent reasoning we want 
to carry out. For example, we may be interested in a wider group of people than only those in 
this class. If we change the domain to consist of all people, we will need to express our statement 
as 


“For every person x, if person x is a student in this class then x has studied calculus.” 


If S(x) represents the statement that person ~x is in this class, we see that our statement can be 
expressed as Vx(S(x) — C(x)). [Caution! Our statement cannot be expressed as Vx(S(x) A 
C(x)) because this statement says that all people are students in this class and have studied 
calculus! ] 

Finally, when we are interested in the background of people in subjects besides calculus, 
we may prefer to use the two-variable quantifier Q(x, y) for the statement “student x has 
studied subject y.” Then we would replace C(x) by Q(x, calculus) in both approaches to obtain 
Vx Q(x, calculus) or Vx(S(x) — Q(x, calculus)). < 
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In Example 23 we displayed different approaches for expressing the same statement using 
predicates and quantifiers. However, we should always adopt the simplest approach that is 
adequate for use in subsequent reasoning. 


Express the statements “Some student in this class has visited Mexico” and “Every student in 
this class has visited either Canada or Mexico” using predicates and quantifiers. 


Solution: The statement “Some student in this class has visited Mexico” means that 
“There is a student in this class with the property that the student has visited Mexico.” 
We can introduce a variable x, so that our statement becomes 
“There is a student x in this class having the property that x has visited Mexico.” 


We introduce M(x), which is the statement “x has visited Mexico.” If the domain for x consists 
of the students in this class, we can translate this first statement as dx M(x). 

However, if we are interested in people other than those in this class, we look at the statement 
a little differently. Our statement can be expressed as 


“There is a person x having the properties that x is a student in this class and x has visited 
Mexico.” 


In this case, the domain for the variable x consists of all people. We introduce S(x) to represent 
“x is a student in this class.” Our solution becomes 4x(S(x) A M(x)) because the statement is 
that there is a person x who is a student in this class and who has visited Mexico. [Caution! Our 
statement cannot be expressed as 4x(S(x) — M(.x)), which is true when there is someone not 
in the class because, in that case, for such a person x, S(x) — M(x) becomes either F > T or 
F — F, both of which are true. ] 

Similarly, the second statement can be expressed as 


“For every x in this class, x has the property that x has visited Mexico or x has visited 
Canada.” 


(Note that we are assuming the inclusive, rather than the exclusive, or here.) We let C(x) be “x 
has visited Canada.” Following our earlier reasoning, we see that if the domain for x consists of 
the students in this class, this second statement can be expressed as Vx (C(x) V M(x)). However, 
if the domain for x consists of all people, our statement can be expressed as 


“For every person x, if x is a student in this class, then x has visited Mexico or x has visited 
Canada.” 


In this case, the statement can be expressed as Vx(S(x) > (C(x) V M(x))). 

Instead of using M(x) and C(x) to represent that x has visited Mexico and x has visited 
Canada, respectively, we could use a two-place predicate V(x, y) to represent “x has visited 
country y.” In this case, V(x, Mexico) and V (x, Canada) would have the same meaning as M (x) 
and C(x) and could replace them in our answers. If we are working with many statements that 
involve people visiting different countries, we might prefer to use this two-variable approach. 
Otherwise, for simplicity, we would stick with the one-variable predicates M(x) and C(x). 4 
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EXAMPLE 26 


Using Quantifiers in System Specifications 


In Section 1.2 we used propositions to represent system specifications. However, many system 
specifications involve predicates and quantifications. This is illustrated in Example 25. 


Use predicates and quantifiers to express the system specifications “Every mail message larger 
than one megabyte will be compressed” and “Tf a user is active, at least one network link will 
be available.” 


Solution: Let S(m, y) be “Mail message m is larger than y megabytes,” where the variable x has 
the domain of all mail messages and the variable y is a positive real number, and let C (m) denote 
“Mail message m will be compressed.” Then the specification “Every mail message larger than 
one megabyte will be compressed” can be represented as Vm(S(m, 1) > C(m)). 

Let A(u) represent “User u is active,’ where the variable u has the domain of all users, 
let S(n, x) denote “Network link n is in state x,’ where n has the domain of all network 
links and x has the domain of all possible states for a network link. Then the specifica- 
tion “If a user is active, at least one network link will be available” can be represented by 
duA(u) > AnS(n, available). < 


Examples from Lewis Carroll 


Lewis Carroll (really C. L. Dodgson writing under a pseudonym), the author of Alice in Wonder- 
land, is also the author of several works on symbolic logic. His books contain many examples 
of reasoning using quantifiers. Examples 26 and 27 come from his book Symbolic Logic; other 
examples from that book are given in the exercises at the end of this section. These examples 
illustrate how quantifiers are used to express various types of statements. 


Consider these statements. The first two are called premises and the third is called the conclusion. 
The entire set is called an argument. 


“All lions are fierce.” 
“Some lions do not drink coffee.” 
“Some fierce creatures do not drink coffee.” 


(In Section 1.6 we will discuss the issue of determining whether the conclusion is a valid conse- 
quence of the premises. In this example, it is.) Let P(x), Q(x), and R(x) be the statements “x is 
a lion,” “x is fierce,” and “x drinks coffee,” respectively. Assuming that the domain consists of all 
creatures, express the statements in the argument using quantifiers and P(x), Q(x), and R(x). 


CHARLES LUTWIDGE DODGSON (1832-1898) We know Charles Dodgson as Lewis Carroll—the 
pseudonym he used in his literary works. Dodgson, the son of a clergyman, was the third of 11 children, 
all of whom stuttered. He was uncomfortable in the company of adults and is said to have spoken without 
stuttering only to young girls, many of whom he entertained, corresponded with, and photographed (sometimes 
in poses that today would be considered inappropriate). Although attracted to young girls, he was extremely 
puritanical and religious. His friendship with the three young daughters of Dean Liddell led to his writing Alice 
in Wonderland, which brought him money and fame. 

Dodgson graduated from Oxford in 1854 and obtained his master of arts degree in 1857. He was appointed 
lecturer in mathematics at Christ Church College, Oxford, in 1855. He was ordained in the Church of England 


in 1861 but never practiced his ministry. His writings published under this real name include articles and books on geometry, 
determinants, and the mathematics of tournaments and elections. (He also used the pseudonym Lewis Carroll for his many works 
on recreational logic.) 
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Solution: We can express these statements as: 


Vx(P(x) > Q(x)). 
Ix(P(x) A =R(x)). 
Ax(Q(x) A 7R(x)). 


Notice that the second statement cannot be written as 4x(P(x) — —R(x)). The reason is that 
P(x) — —R(x) is true whenever x is not a lion, so that 3x(P(x) > —R(x)) is true as long as 
there is at least one creature that is not a lion, even if every lion drinks coffee. Similarly, the 
third statement cannot be written as 


Ix(Q(x) > R(x). < 


Consider these statements, of which the first three are premises and the fourth is a valid conclu- 
sion. 


“All hummingbirds are richly colored.” 

“No large birds live on honey.” 

“Birds that do not live on honey are dull in color.” 
“Hummingbirds are small.” 


Let P(x), Q(x), R(x), and S(x) be the statements “x is ahummingbird,” “x is large,” “x lives on 
honey,” and “‘x is richly colored,” respectively. Assuming that the domain consists of all birds, 
express the statements in the argument using quantifiers and P(x), Q(x), R(x), and S(x). 


Solution: We can express the statements in the argument as 


Vx (P(x) > S(x)). 
74x (Q(x) A R(x)). 
Vx(-R(x) > 7S(x)). 
Wx(P(x) > 7Q(x)). 


(Note we have assumed that “small” is the same as “not large” and that “dull in color” is the 
same as “not richly colored.” To show that the fourth statement is a valid conclusion of the first 
three, we need to use rules of inference that will be discussed in Section 1.6.) < 


Logic Programming 


An important type of programming language is designed to reason using the rules of predicate 
logic. Prolog (from Programming in Logic), developed in the 1970s by computer scientists 
working in the area of artificial intelligence, is an example of such a language. Prolog programs 
include a set of declarations consisting of two types of statements, Prolog facts and Prolog 
rules. Prolog facts define predicates by specifying the elements that satisfy these predicates. 
Prolog rules are used to define new predicates using those already defined by Prolog facts. 
Example 28 illustrates these notions. 


Consider a Prolog program given facts telling it the instructor of each class and in which classes 
students are enrolled. The program uses these facts to answer queries concerning the professors 
who teach particular students. Such a program could use the predicates instructor(p,c) and 
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enrolled(s, c) to represent that professor p is the instructor of course c and that student s 
is enrolled in course c, respectively. For example, the Prolog facts in such a program might 
include: 


instructor (chan,math273) 
instructor (patel, ee222) 
instructor (grossman,cs301) 
enrolled(kevin,math273) 
enrolled (juana,ee222) 
enrolled (juana,cs301) 
enrolled(kiko,math273) 
enrolled(kiko,cs301) 


(Lowercase letters have been used for entries because Prolog considers names beginning with 
an uppercase letter to be variables.) 

A new predicate teaches(p,s), representing that professor p teaches student s, can be 
defined using the Prolog rule 


teaches(P,S) :- instructor(P,C), enrolled(S,C) 


which means that teaches(p,s) is true if there exists a class c such that professor p is the 
instructor of class c and student s is enrolled in class c. (Note that a comma is used to represent 
a conjunction of predicates in Prolog. Similarly, a semicolon is used to represent a disjunction 
of predicates.) 

Prolog answers queries using the facts and rules it is given. For example, using the facts 
and rules listed, the query 


Penrolled(kevin,math273) 


produces the response 


yes 


because the fact enrolled(kevin, math273) was provided as input. The query 


Penrolled(X,math273) 


produces the response 


kevin 
kiko 


To produce this response, Prolog determines all possible values of X for which 
enrolled(X , math273) has been included as a Prolog fact. Similarly, to find all the professors 
who are instructors in classes being taken by Juana, we use the query 


?teaches (X, juana) 


This query returns 


patel <q 
grossman 


Exercises 


. Let P(x) denote the statement “x < 4’ What are these 

truth values? 

a) P(O) b) P(4) c) P(6) 

. Let P(x) be the statement “the word x contains the 

letter a.” What are these truth values? 

a) P(orange) b) P(lemon) 

c) P(true) d) P(false) 

. Let Q(x, y) denote the statement “x is the capital of y.” 

What are these truth values? 

a) Q(Denver, Colorado) 

b) Q(Detroit, Michigan) 

c) Q(Massachusetts, Boston) 

d) Q(New York, New York) 

. State the value of x after the statement if P(x) thenx := 1 

is executed, where P(x) is the statement “x > 1,” if the 

value of x when this statement is reached is 

a) x=0. b) x =1. 

c) x =2. 

. Let P(x) be the statement “x spends more than five hours 

every weekday in class,” where the domain for x consists 

of all students. Express each of these quantifications in 

English. 

a) dx P(x) b) Vx P(x) 

c) Ax -P(x) d) Vx =P(x) 

. Let N(x) be the statement “x has visited North Dakota,” 

where the domain consists of the students in your school. 

Express each of these quantifications in English. 

a) AxN(x) b) Vx N(x) ce) 7Ax N(x) 

d) Ax-N (x) e) -VxN(x) f) Vx-N(x) 

. Translate these statements into English, where C (x) is “x 

is acomedian” and F(x) is “x is funny” and the domain 

consists of all people. 

a) Vx(C(x) > F(x)) b) Vx(C(x) A F(x)) 

ce) dx(C(x) > F(x)) d) Ax(C(x) A F(x)) 

. Translate these statements into English, where R(x) is “x 

is arabbit” and H (x) is “x hops” and the domain consists 

of all animals. 

a) Vx(R(x) > A(x)) b) Vx(R(x) A A(x)) 

c) Ax(R(x) > A(x)) d) Ax(R(x) A AH(x)) 

. Let P(x) be the statement “x can speak Russian” and let 

Q(x) be the statement “x knows the computer language 

C++.” Express each of these sentences in terms of P(x), 

Q(x), quantifiers, and logical connectives. The domain 

for quantifiers consists of all students at your school. 

a) There is a student at your school who can speak Rus- 
sian and who knows C++. 

b) There is a student at your school who can speak Rus- 
sian but who doesn’t know C++. 

c) Every student at your school either can speak Russian 
or knows C++. 

d) No student at your school can speak Russian or knows 
C++. 
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Let C(x) be the statement “x has a cat,’ let D(x) be the 

statement “x has a dog,” and let F(x) be the statement “x 

has a ferret.” Express each of these statements in terms of 

C(x), D(x), F(x), quantifiers, and logical connectives. 

Let the domain consist of all students in your class. 

a) A student in your class has a cat, a dog, and a ferret. 

b) All students in your class have a cat, a dog, or a ferret. 

c) Some student in your class has a cat and a ferret, but 
not a dog. 

d) No student in your class has a cat, a dog, and a ferret. 

e) For each of the three animals, cats, dogs, and ferrets, 
there is a student in your class who has this animal as 
a pet. 

Let P(x) be the statement “x = x2.” If the domain con- 

sists of the integers, what are these truth values? 

a) P(O) b) P(1) c) P(2) 

d) P(-1) e) Ax P(x) f) Vx P(x) 

Let Q(x) be the statement “x + 1 > 2x.” If the domain 

consists of all integers, what are these truth values? 


a) Q(0) b) Q(-1) c) QC) 
d) Ax Q(x) e) Vx Q(x) f) 4x-Q(x) 
g) VWx7Q(x) 


Determine the truth value of each of these statements if 
the domain consists of all integers. 

a) Vn(n+1>n) b) An(2n = 3n) 

c) dn(n = —n) d) Vn(3n < 4n) 
Determine the truth value of each of these statements if 
the domain consists of all real numbers. 

a) dx(x? = —1) b) Ax(x* < x?) 

c) Wx((—x)? = x?) d) Vx(2x > x) 

Determine the truth value of each of these statements if 
the domain for all variables consists of all integers. 

a) Yn(n* > 0) b) An(n? = 2) 

c) Wn(n2 > n) d) dn(n?2 < 0) 

Determine the truth value of each of these statements if 
the domain of each variable consists of all real numbers. 

a) dx(x? = 2) b) Ax(x? = —1) 

c) Wx(x2+2> 1) d) Vx(x? 4 x) 

Suppose that the domain of the propositional function 
P(x) consists of the integers 0, 1, 2, 3, and 4. Write out 
each of these propositions using disjunctions, conjunc- 
tions, and negations. 

a) dx P(x) b) Vx P(x) c) Ax-P(x) 

d) Vx-P(x) e) 74x P(x) f) -Vx P(x) 
Suppose that the domain of the propositional function 
P(x) consists of the integers —2, —1, 0, 1, and 2. Write 
out each of these propositions using disjunctions, con- 
junctions, and negations. 


a) dx P(x) b) Vx P(x) 
d) Vx-P(x) e) 74x P(x) 


c) dax-P(x) 
f) -Vx P(x) 
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Suppose that the domain of the propositional function 
P(x) consists of the integers 1, 2, 3, 4, and 5. Express 
these statements without using quantifiers, instead using 
only negations, disjunctions, and conjunctions. 

a) Ax P(x) b) Vx P(x) 

ce) dx P(x) d) -Vx P(x) 

e) Vx((x #3) > P(x)) VAx-P(x) 

Suppose that the domain of the propositional function 
P(x) consists of —5, —3, —1, 1, 3, and 5. Express these 
statements without using quantifiers, instead using only 
negations, disjunctions, and conjunctions. 

a) dx P(x) b) Vx P(x) 

c) Vx((~ # 1) > P(x)) 

d) Ax((x = 0) A P(x)) 

e) Ax(4P(x)) AVx((x < 0) > P(x)) 

For each of these statements find a domain for which the 
statement is true and a domain for which the statement is 
false. 


a) Everyone is studying discrete mathematics. 

b) Everyone is older than 21 years. 

c) Every two people have the same mother. 

d) No two different people have the same grandmother. 
For each of these statements find a domain for which the 
statement is true and a domain for which the statement is 
false. 


a) Everyone speaks Hindi. 

b) There is someone older than 21 years. 

c) Every two people have the same first name. 
d) Someone knows more than two other people. 


Translate in two ways each of these statements into logi- 
cal expressions using predicates, quantifiers, and logical 
connectives. First, let the domain consist of the students 
in your class and second, let it consist of all people. 


a) Someone in your class can speak Hindi. 

b) Everyone in your class is friendly. 

c) There is a person in your class who was not born in 
California. 

d) A student in your class has been in a movie. 

e) No student in your class has taken a course in logic 
programming. 

Translate in two ways each of these statements into logi- 

cal expressions using predicates, quantifiers, and logical 

connectives. First, let the domain consist of the students 

in your class and second, let it consist of all people. 


a) Everyone in your class has a cellular phone. 

b) Somebody in your class has seen a foreign movie. 

c) There is a person in your class who cannot swim. 

d) All students in your class can solve quadratic equa- 
tions. 

e) Some student in your class does not want to be rich. 


Translate each of these statements into logical expres- 
sions using predicates, quantifiers, and logical connec- 
tives. 

a) No one is perfect. 

b) Not everyone is perfect. 

c) All your friends are perfect. 

d) At least one of your friends is perfect. 
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e) Everyone is your friend and is perfect. 

f) Not everybody is your friend or someone is not per- 
fect. 

Translate each of these statements into logical expres- 

sions in three different ways by varying the domain and 

by using predicates with one and with two variables. 

a) Someone in your school has visited Uzbekistan. 

b) Everyone in your class has studied calculus and C++. 

c) No one in your school owns both a bicycle and a mo- 
torcycle. 

d) There is a person in your school who is not happy. 

e) Everyone in your school was born in the twentieth 
century. 

Translate each of these statements into logical expres- 

sions in three different ways by varying the domain and 

by using predicates with one and with two variables. 

a) A student in your school has lived in Vietnam. 

b) There is a student in your school who cannot speak 
Hindi. 

c) A student in your school knows Java, Prolog, and 
C++. 

d) Everyone in your class enjoys Thai food. 

e) Someone in your class does not play hockey. 

Translate each of these statements into logical expres- 

sions using predicates, quantifiers, and logical connec- 

tives. 

a) Something is not in the correct place. 

b) All tools are in the correct place and are in excellent 
condition. 

c) Everything is in the correct place and in excellent con- 
dition. 

d) Nothing is in the correct place and is in excellent con- 
dition. 

e) One of your tools is not in the correct place, but it is 
in excellent condition. 

Express each of these statements using logical operators, 

predicates, and quantifiers. 

a) Some propositions are tautologies. 

b) The negation of a contradiction is a tautology. 

c) The disjunction of two contingencies can be a tautol- 
ogy. 

d) The conjunction of two tautologies is a tautology. 

Suppose the domain of the propositional function P(x, y) 

consists of pairs x and y, where x is 1, 2, or 3 and y is 

1, 2, or 3. Write out these propositions using disjunctions 

and conjunctions. 

a) Ax P(x, 3) b) Vy Pd, y) 

ce) dy7P(2, y) d) Vx —P(x, 2) 

Suppose that the domain of Q(x, y, z) consists of triples 

x,y,z, where x = 0,1, or 2, y=Oorl, andz=Oor1. 

Write out these propositions using disjunctions and con- 

junctions. 

a) VyQ(0, y, 0) 

c) 4z-Q(0, 0, z) 


b) 4x Q(x, 1, 1) 
d) Ax-Q(x, 0, 1) 
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Express each of these statements using quantifiers. Then 

form the negation of the statement so that no negation is 

to the left of a quantifier. Next, express the negation in 

simple English. (Do not simply use the phrase “It is not 

the case that.’’) 

a) All dogs have fleas. 

b) There is a horse that can add. 

c) Every koala can climb. 

d) No monkey can speak French. 

e) There exists a pig that can swim and catch fish. 

Express each of these statements using quantifiers. Then 

form the negation of the statement, so that no negation 

is to the left of a quantifier. Next, express the negation in 

simple English. (Do not simply use the phrase “It is not 

the case that.”) 

a) Some old dogs can learn new tricks. 

b) No rabbit knows calculus. 

c) Every bird can fly. 

d) There is no dog that can talk. 

e) There is no one in this class who knows French and 
Russian. 

Express the negation of these propositions using quanti- 

fiers, and then express the negation in English. 

a) Some drivers do not obey the speed limit. 

b) All Swedish movies are serious. 

c) No one can keep a secret. 

d) There is someone in this class who does not have a 
good attitude. 

Find a counterexample, if possible, to these universally 

quantified statements, where the domain for all variables 

consists of all integers. 

a) Wx(x2 > x) 

b) Vx(x > OV x <0) 

c) Vx(x = 1) 

Find a counterexample, if possible, to these universally 


quantified statements, where the domain for all variables 

consists of all real numbers. 

a) Wx(x2 # x) 

c) Vx(|x| > 0) 

Express each of these statements using predicates and 

quantifiers. 

a) A passenger on an airline qualifies as an elite flyer if 
the passenger flies more than 25,000 miles in a year 
or takes more than 25 flights during that year. 

b) A man qualifies for the marathon if his best previ- 
ous time is less than 3 hours and a woman qualifies 
for the marathon if her best previous time is less than 
3.5 hours. 

c) A student must take at least 60 course hours, or at least 
45 course hours and write a master’s thesis, and re- 
ceive a grade no lower than a B in all required courses, 
to receive a master’s degree. 

d) There is a student who has taken more than 21 credit 
hours in a semester and received all A’s. 


b) Vx(x? 4 2) 
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Exercises 38-42 deal with the translation between system 
specification and logical expressions involving quantifiers. 


38. 
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Translate these system specifications into English where 

the predicate S(x, y) is “x is in state y” and where the 

domain for x and y consists of all systems and all possible 

states, respectively. 

a) dxS(x, open) 

b) Vx(S(x, malfunctioning) Vv S(x, diagnostic)) 

c) AxS(x, open) Vv 4x S(x, diagnostic) 

d) Ax—-S(x, available) 

e) Vx-=S(x, working) 

Translate these specifications into English where F'(p) is 

“Printer p is out of service,’ B(p) is “Printer p is busy,” 

L(j) is “Print job 7 is lost,’ and Q(j) is “Print job 7 is 

queued.” 

a) dp(F(p) A B(p)) > AJL) 

b) VpB(p) > AjQ@(y) 

ce) Aj(QG) A LG)) > ApF(p) 

d) (VpB(p) AVIQ(V)) > AJL) 

Express each of these system specifications using predi- 

cates, quantifiers, and logical connectives. 

a) When there is less than 30 megabytes free on the hard 
disk, a warning message is sent to all users. 

b) No directories in the file system can be opened and 
no files can be closed when system errors have been 
detected. 


c) The file system cannot be backed up if there is a user 
currently logged on. 


d) Video on demand can be delivered when there are at 
least 8 megabytes of memory available and the con- 
nection speed is at least 56 kilobits per second. 

Express each of these system specifications using predi- 

cates, quantifiers, and logical connectives. 

a) At least one mail message, among the nonempty set 
of messages, can be saved if there is a disk with more 
than 10 kilobytes of free space. 


b) Whenever there is an active alert, all queued messages 
are transmitted. 

c) The diagnostic monitor tracks the status of all systems 
except the main console. 

d) Each participant on the conference call whom the host 
of the call did not put on a special list was billed. 
Express each of these system specifications using predi- 

cates, quantifiers, and logical connectives. 

a) Every user has access to an electronic mailbox. 

b) The system mailbox can be accessed by everyone in 
the group if the file system is locked. 

c) The firewall is in a diagnostic state only if the proxy 
server is in a diagnostic state. 

d) At least one router is functioning normally if the 


throughput is between 100 kbps and 500 kbps and 
the proxy server is not in diagnostic mode. 
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Determine whether Vx(P(x) > Q(x)) and Vx P(x) > 
Vx Q(x) are logically equivalent. Justify your answer. 


Determine whether Vx(P(x) < Q(x)) and Vx P(x) 
Vx Q(x) are logically equivalent. Justify your answer. 


Show that 4x (P(x) V Q(x)) and 4x P(x) V Ax Q(x) are 
logically equivalent. 


Exercises 46-49 establish rules for null quantification that 
we can use when a quantified variable does not appear in part 
of a statement. 
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Establish these logical equivalences, where x does not 
occur as a free variable in A. Assume that the domain is 
nonempty. 

a) (Vx P(x)) V A = Vx(P(x) Vv A) 

b) (Ax P(x)) V A = Ax(P(x) V A) 

Establish these logical equivalences, where x does not 
occur as a free variable in A. Assume that the domain is 
nonempty. 

a) (Wx P(x)) A A = Vx(P(x) A A) 

b) (Ax P(x)) A A = Ax(P(x) A A) 

Establish these logical equivalences, where x does not 
occur as a free variable in A. Assume that the domain is 
nonempty. 

a) Wx(A > P(x)) =A — Vx P(x) 

b) ax(A > P(x)) = A —> Ax P(x) 

Establish these logical equivalences, where x does not 
occur as a free variable in A. Assume that the domain is 
nonempty. 

a) Wx(P(x) > A) =4AxP(x) > A 

b) dax(P(x) — A) =VxP(x) > A 

Show that Vx P(x) V Vx Q(x) and Vx(P(x) V Q(x)) are 
not logically equivalent. 

Show that dx P(x) A dx Q(x) and Ax(P(x) A Q(x)) are 
not logically equivalent. 


As mentioned in the text, the notation 4!x P(x) denotes 
“There exists a unique x such that P(x) is true.” 

If the domain consists of all integers, what are the truth 

values of these statements? 

a) dlx(x > 1) b) Alx(x? = 1) 

ce) Alx(x +3 = 2x) d) Alx(x =x+1) 

What are the truth values of these statements? 

a) Alx P(x) > Ax P(x) 

b) Vx P(x) > Alx P(x) 

ce) AlxaP(x) > 7AVx P(x) 

Write out 4!x P(x), where the domain consists of the in- 

tegers 1, 2, and 3, in terms of negations, conjunctions, 

and disjunctions. 

Given the Prolog facts in Example 28, what would Prolog 

return given these queries? 

a) ?instructor(chan,math273) 

b) Pinstructor(patel,cs301) 

c) Penrolled(X,cs301) 

d) ?enrolled(kiko,Y) 

e) ?teaches (grossman, Y) 
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. Given the Prolog facts in Example 28, what would Prolog 


return when given these queries? 

a) Penrolled(kevin, ee222) 

b) ?enrolled(kiko,math273) 

c) Pinstructor(grossman, X) 

d) Pinstructor (X,cs301) 

e) ?teaches (X, kevin) 

Suppose that Prolog facts are used to define the predicates 
mother(M, Y) and father(F, X), which represent that M 
is the mother of Y and F is the father of X, respectively. 
Give a Prolog rule to define the predicate sibling(X, Y), 
which represents that X and Y are siblings (that is, have 
the same mother and the same father). 


Suppose that Prolog facts are used to define the predi- 
cates mother(M, Y) and father(F, X), which represent 
that M is the mother of Y and F is the father of X, 
respectively. Give a Prolog rule to define the predicate 
grandfather(X, Y), which represents that X is the grand- 
father of Y. [Hint: You can write a disjunction in Prolog 
either by using a semicolon to separate predicates or by 
putting these predicates on separate lines.] 


Exercises 59-62 are based on questions found in the book 
Symbolic Logic by Lewis Carroll. 
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. Let P(x), Q(x), and R(x) be the statements “x is a 

professor,” “x is ignorant,’ and “x is vain,” respectively. 
Express each of these statements using quantifiers; log- 
ical connectives; and P(x), Q(x), and R(x), where the 
domain consists of all people. 

a) No professors are ignorant. 

b) All ignorant people are vain. 

c) No professors are vain. 

d) Does (c) follow from (a) and (b)? 

Let P(x), O(x), and R(x) be the statements “‘x is a clear 

explanation,” “x is satisfactory,’ and “x is an excuse,” 
respectively. Suppose that the domain for x consists of all 
English text. Express each of these statements using quan- 
tifiers, logical connectives, and P(x), Q(x), and R(x). 

a) All clear explanations are satisfactory. 
b) Some excuses are unsatisfactory. 
c) Some excuses are not clear explanations. 

*d) Does (c) follow from (a) and (b)? 

. Let P(x), Q(x), R(x), and S(x) be the statements “x is 
a baby,” “x is logical,” “‘x is able to manage a crocodile,” 
and “x is despised,” respectively. Suppose that the domain 
consists of all people. Express each of these statements 
using quantifiers; logical connectives; and P(x), Q(x), 
R(x), and S(x). 

a) Babies are illogical. 
b) Nobody is despised who can manage a crocodile. 
c) Illogical persons are despised. 
d) Babies cannot manage crocodiles. 
*e) Does (d) follow from (a), (b), and (c)? If not, is there 
a correct conclusion? 
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62. Let P(x), Q(x), R(x), and S(x) be the statements “x b) No officers ever decline to waltz. 


is a duck,” “x is one of my poultry, 


” “x is an officer,” c) All my poultry are ducks. 


and “‘x is willing to waltz,” respectively. Express each of 


these statements using quantifiers; logical connectives; d) My poultry are not officers. 
and P(x), Q(x), R(x), and S(x). *e) Does (d) follow from (a), (b), and (c)? If not, is there 
a) No ducks are willing to waltz. a correct conclusion? 


Nested Quantifiers 


EXAMPLE 1 


Extra 
Examples 


é 


Introduction 


In Section 1.4 we defined the existential and universal quantifiers and showed how they can 
be used to represent mathematical statements. We also explained how they can be used to 
translate English sentences into logical expressions. However, in Section 1.4 we avoided nested 
quantifiers, where one quantifier is within the scope of another, such as 


Vxdy(x + y = 0). 


Note that everything within the scope of a quantifier can be thought of as a propositional function. 
For example, 


Vxdy(x + y = 0) 


is the same thing as Vx Q(x), where Q(x) is dyP(x, y), where P(x, y)isx+y=0. 

Nested quantifiers commonly occur in mathematics and computer science. Although nested 
quantifiers can sometimes be difficult to understand, the rules we have already studied in 
Section 1.4 can help us use them. In this section we will gain experience working with nested 
quantifiers. We will see how to use nested quantifiers to express mathematical statements such 
as “The sum of two positive integers is always positive.” We will show how nested quantifiers 
can be used to translate English sentences such as “Everyone has exactly one best friend” into 
logical statements. Moreover, we will gain experience working with the negations of statements 
involving nested quantifiers. 


Understanding Statements Involving Nested Quantifiers 


To understand statements involving nested quantifiers, we need to unravel what the quantifiers 
and predicates that appear mean. This is illustrated in Examples | and 2. 


Assume that the domain for the variables x and y consists of all real numbers. The statement 
VxVy(x+y=y+x) 


says that x + y = y + x for all real numbers x and y. This is the commutative law for addition 
of real numbers. Likewise, the statement 


Vxdy(x + y = 0) 


says that for every real number x there is a real number y such that x + y = 0. This states that 
every real number has an additive inverse. Similarly, the statement 


VxVyVz(x + (y +z) = (w+ y) +2) 


is the associative law for addition of real numbers. < 
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EXAMPLE 2 


EXAMPLE 3 


Extra p> 
Examples Qi 


Translate into English the statement 
VxVy((x > 0) A (y < 0) > (xy < 0)), 
where the domain for both variables consists of all real numbers. 


Solution: This statement says that for every real number x and for every real number y, if x > 0 
and y < 0, then xy < 0. That is, this statement says that for real numbers x and y, if x is positive 
and y is negative, then xy is negative. This can be stated more succinctly as “The product of a 
positive real number and a negative real number is always a negative real number.” < 


THINKING OF QUANTIFICATION AS LOOPS In working with quantifications of more 
than one variable, it is sometimes helpful to think in terms of nested loops. (Of course, if there 
are infinitely many elements in the domain of some variable, we cannot actually loop through 
all values. Nevertheless, this way of thinking is helpful in understanding nested quantifiers.) For 
example, to see whether VxVy P(x, y) is true, we loop through the values for x, and for each x 
we loop through the values for y. If we find that P(x, y) is true for all values for x and y, we 
have determined that VxVy P(x, y) is true. If we ever hit a value x for which we hit a value y 
for which P(x, y) is false, we have shown that VxVy P(x, y) is false. 

Similarly, to determine whether Vxdy P(x, y) is true, we loop through the values for x. 
For each x we loop through the values for y until we find a y for which P(x, y) is true. If for 
every x we hit such a y, then Vxdy P(x, y) is true; if for some x we never hit such a y, then 
Vxdy P(x, y) is false. 

To see whether 4xVy P(x, y) is true, we loop through the values for x until we find an x for 
which P(x, y) is always true when we loop through all values for y. Once we find such an x, we 
know that dxVy P(x, y) is true. If we never hit such an x, then we know that 4xVy P(x, y) is false. 

Finally, to see whether xy P(x, y) is true, we loop through the values for x, where for 
each x we loop through the values for y until we hit an x for which we hit a y for which P(x, y) 
is true. The statement dxdy P(x, y) is false only if we never hit an x for which we hit a y such 
that P(x, y) is true. 


The Order of Quantifiers 


Many mathematical statements involve multiple quantifications of propositional functions in- 
volving more than one variable. It is important to note that the order of the quantifiers is important, 
unless all the quantifiers are universal quantifiers or all are existential quantifiers. 

These remarks are illustrated by Examples 3-S. 


Let P(x, y) be the statement “x + y = y + x.” What are the truth values of the quantifications 
VxVy P(x, y) and VyVx P(x, y) where the domain for all variables consists of all real numbers? 


Solution: The quantification 

VxVy P(x, y) 
denotes the proposition 

“For all real numbers x, for all real numbers y, x + y= y+ x.” 
Because P(x, y) 1s true for all real numbers x and y (it is the commutative law for addition, 
which is an axiom for the real numbers—see Appendix 1), the proposition VxVy P(x, y) is 
true. Note that the statement VyVx P(x, y) says “For all real numbers y, for all real numbers x, 


x+y = y+-x.’ This has the same meaning as the statement “For all real numbers x, for all real 
numbers y, x + y= y+ x.” That is, VxVy P(x, y) and VyVx P(x, y) have the same meaning, 


EXAMPLE 4 


Be careful with the order 
of existential and 
universal quantifiers! 


EXAMPLE 5 
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and both are true. This illustrates the principle that the order of nested universal quantifiers 
in a statement without other quantifiers can be changed without changing the meaning of the 
quantified statement. 


Let Q(x, y) denote “x + y = 0.” What are the truth values of the quantifications JyVx Q(x, y) 
and Vxdy Q(x, y), where the domain for all variables consists of all real numbers? 


Solution: The quantification 
dyVx Q(x, y) 
denotes the proposition 
“There is areal number y such that for every real number x, Q(x, y).” 


No matter what value of y is chosen, there is only one value of x for which x + y = 0. Because 
there is no real number y such that x + y = 0 for all real numbers x, the statement JyVx Q(x, y) 
is false. 

The quantification 


VxAy Q(x, y) 
denotes the proposition 
“For every real number x there is a real number y such that Q(x, y).” 


Given a real number x, there is a real number y such that x + y = 0; namely, y = —x. Hence, 
the statement Vxdy Q(x, y) is true. 


Example 4 illustrates that the order in which quantifiers appear makes a difference. The state- 
ments 4yVx P(x, y) and Vxdy P(x, y) are not logically equivalent. The statement SyVx P(x, y) 
is true if and only if there is a y that makes P(x, y) true for every x. So, for this statement to 
be true, there must be a particular value of y for which P(x, y) is true regardless of the choice 
of x. On the other hand, Vxdy P(x, y) 1s true if and only if for every value of x there is a value 
of y for which P(x, y) is true. So, for this statement to be true, no matter which x you choose, 
there must be a value of y (possibly depending on the x you choose) for which P(x, y) is true. 
In other words, in the second case, y can depend on x, whereas in the first case, y is a constant 
independent of x. 

From these observations, it follows that if JyVx P(x, y) is true, then Vxdy P(x, y) must 
also be true. However, if Vxiay P(x, y) is true, it is not necessary for JyVx P(x, y) to be true. 
(See Supplementary Exercises 30 and 31.) 

Table 1 summarizes the meanings of the different possible quantifications involving two 
variables. 

Quantifications of more than two variables are also common, as Example 5 illustrates. 


Let Q(x, y, z) be the statement “x + y = z.’ What are the truth values of the statements 
VxVydzQO(x, y, z) and 4zVxVy Q(x, y, z), where the domain of all variables consists of all 
real numbers? 


Solution: Suppose that x and y are assigned values. Then, there exists a real number z such that 
x + y =z. Consequently, the quantification 


VxVydzO(x, y, Z), 
which is the statement 


“For all real numbers x and for all real numbers y there is a real number z such that 
x+y =Z,” 
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EXAMPLE 6 


Extra p> 
Examples 2 


EXAMPLE 7 


TABLE 1 Quantifications of Two Variables. 


VxVy P(x, y) P(x, y) is true for every pair x, y. There is a pair x, y for 
VyVx P(x, y) which P(x, y) is false. 


Vxdy P(x, y) For every x there is a y for There is an x such that 
which P(x, y) is true. P(x, y) is false for every y. 
AxVy P(x, y) There is an x for which P(x, y) For every x there is a y for 
is true for every y. which P(x, y) is false. 
Axdy P(x, y) There is a pair x, y for which P(x, y) is false for every 
dydx P(x, y) P(x, y) is true. pair x, y. 


is true. The order of the quantification here is important, because the quantification 
AzVxVy Q(x, y, Z), 


which is the statement 


“There is a real number z such that for all real numbers x and for all real numbers y it is 
true that x + y = z,” 


is false, because there is no value of z that satisfies the equation x + y = z for all values of x 
and y. < 


Translating Mathematical Statements into Statements 
Involving Nested Quantifiers 


Mathematical statements expressed in English can be translated into logical expressions, as 
Examples 6-8 show. 


Translate the statement “The sum of two positive integers is always positive” into a logical 
expression. 


Solution: To translate this statement into a logical expression, we first rewrite it so that the implied 
quantifiers and a domain are shown: “For every two integers, if these integers are both positive, 
then the sum of these integers is positive.’ Next, we introduce the variables x and y to obtain “For 
all positive integers x and y, x + y is positive.” Consequently, we can express this statement as 


VxVy((x > 0) A (y > 0) > (4+ y > 0)), 


where the domain for both variables consists of all integers. Note that we could also translate 
this using the positive integers as the domain. Then the statement “The sum of two positive 
integers is always positive” becomes “For every two positive integers, the sum of these integers 
is positive. We can express this as 


VxVy(x + y > 0), 


where the domain for both variables consists of all positive integers. < 


Translate the statement “Every real number except zero has a multiplicative inverse.” (A mul- 
tiplicative inverse of a real number x is a real number y such that xy = 1.) 


EXAMPLE 8 


EXAMPLE 9 


EXAMPLE 10 
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Solution: We first rewrite this as “For every real number x except zero, x has a multiplicative 
inverse.” We can rewrite this as “For every real number x, if x 4 0, then there exists a real 
number y such that xy = 1.” This can be rewritten as 


Vx((x #0) > Ay(xy = 1)). < 


One example that you may be familiar with is the concept of limit, which is important in 
calculus. 


(Requires calculus) Use quantifiers to express the definition of the limit of a real-valued 
function f(x) of a real variable x at a point a in its domain. 


Solution: Recall that the definition of the statement 
lim f(x) =L 
xa 


is: For every real number € > 0 there exists a real number 6 > O such that | f(x) — L| <«€ 
whenever 0 < |x — a| < 6. This definition of a limit can be phrased in terms of quantifiers by 


VedbVx(0 < |x —al <b > |f(x) -—L| <6), 


where the domain for the variables 6 and € consists of all positive real numbers and for x consists 
of all real numbers. 
This definition can also be expressed as 


Ve >055 >0Vx(0 < |x —al <5 > | f(x) -—L| < €) 


when the domain for the variables € and 6 consists of all real numbers, rather than just the positive 
real numbers. [Here, restricted quantifiers have been used. Recall that Vx >0O P(x) means that 
for all x with x> 0, P(x) is true.] 


Translating from Nested Quantifiers into English 


Expressions with nested quantifiers expressing statements in English can be quite complicated. 
The first step in translating such an expression is to write out what the quantifiers and predicates 
in the expression mean. The next step is to express this meaning in a simpler sentence. This 
process is illustrated in Examples 9 and 10. 


Translate the statement 
Vx(C(x) V Ay(C(y) A F(x, y))) 


into English, where C(x) is “x has a computer,” F(x, y) is “x and y are friends,” and the domain 
for both x and y consists of all students in your school. 


Solution: The statement says that for every student x in your school, x has a computer or there 


is a student y such that y has a computer and x and y are friends. In other words, every student 
in your school has a computer or has a friend who has a computer. 


Translate the statement 


AxVyVz2((F (x, y) A F(x, 2) A (y # Z)) @ AF Q,Z)) 
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EXAMPLE 11 


EXAMPLE 12 


into English, where F'(a,b) means a and D are friends and the domain for x, y, and z consists of 
all students in your school. 


Solution: We first examine the expression (F(x, y) A F(x, z) A (y # z)) > 7-FQ, z). This 
expression says that if students x and y are friends, and students x and z are friends, and 
furthermore, if y and z are not the same student, then y and z are not friends. It follows that 
the original statement, which is triply quantified, says that there is a student x such that for all 
students y and all students z other than y, if x and y are friends and x and z are friends, then y 
and z are not friends. In other words, there is a student none of whose friends are also friends 
with each other. < 


Translating English Sentences into Logical Expressions 


In Section 1.4 we showed how quantifiers can be used to translate sentences into logical expres- 
sions. However, we avoided sentences whose translation into logical expressions required the 
use of nested quantifiers. We now address the translation of such sentences. 


Express the statement “If a person is female and is a parent, then this person is someone’s 
mother” as a logical expression involving predicates, quantifiers with a domain consisting of all 
people, and logical connectives. 


Solution: The statement “If a person is female and is a parent, then this person is someone’s 
mother” can be expressed as “For every person x, if person x is female and person x is a parent, 
then there exists a person y such that person x is the mother of person y.” We introduce the 
propositional functions F(x) to represent “x is female,’ P(x) to represent “‘x is a parent,” and 
M(x, y) to represent “x is the mother of y.” The original statement can be represented as 


Vx((F (x) A P(x)) > AyM (x, y)). 


Using the null quantification rule in part (b) of Exercise 47 in Section 1.4, we can move Sy to 
the left so that it appears just after Vx, because y does not appear in F(x) A P(x). We obtain 
the logically equivalent expression 


Vxdy((F (x) A P(x)) > M(x, y)). < 


Express the statement “Everyone has exactly one best friend” as a logical expression involving 
predicates, quantifiers with a domain consisting of all people, and logical connectives. 


Solution: The statement “Everyone has exactly one best friend” can be expressed as “For every 
person x, person x has exactly one best friend.” Introducing the universal quantifier, we see 
that this statement is the same as “Vx(person x has exactly one best friend),’ where the domain 
consists of all people. 

To say that x has exactly one best friend means that there is a person y who is the best friend 
of x, and furthermore, that for every person z, if person z is not person y, then z is not the best 
friend of x. When we introduce the predicate B(x, y) to be the statement “‘y is the best friend 
of x,” the statement that x has exactly one best friend can be represented as 


Ay(B(x, y) AVZ(Zz # y) > “BO, z))). 
Consequently, our original statement can be expressed as 


Vxdy(B(x, y) AVz((z # y) > =B(x, z))). 


EXAMPLE 13 


Assessment 


EXAMPLE 14 
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[Note that we can write this statement as Vxi!y B(x, y), where 3! is the “uniqueness quantifier” 
defined in Section 1.4.] 


Use quantifiers to express the statement “There is a woman who has taken a flight on every 
airline in the world.” 


Solution: Let P(w, f) be “w has taken f” and Q(f, a) be “f is a flight on a.” We can express 
the statement as 


Iwas f (P(w, f) A Q(f,a)), 


where the domains of discourse for w, f, and a consist of all the women in the world, all airplane 
flights, and all airlines, respectively. 
The statement could also be expressed as 


AwVai f R(w, f, a), 


where R(w, f, a) is “w has taken f ona.” Although this is more compact, it somewhat obscures 
the relationships among the variables. Consequently, the first solution is usually preferable. 4 


Negating Nested Quantifiers 


Statements involving nested quantifiers can be negated by successively applying the rules for 
negating statements involving a single quantifier. This is illustrated in Examples 14-16. 


Express the negation of the statement Vx y(xy = 1) so that no negation precedes a quantifier. 


Solution: By successively applying De Morgan’s laws for quantifiers in Table 2 of 
Section 1.4, we can move the negation in ~Vxdy(xy = 1) inside all the quantifiers. We find 
that ~Vxdy(xy = 1) is equivalent to 3x-Ay(xy = 1), which is equivalent to IxVy7(xy = 1). 
Because —(xy = 1) can be expressed more simply as xy 4 1, we conclude that our negated 
statement can be expressed as 4xVy(xy 4 1). < 


Use quantifiers to express the statement that “There does not exist a woman who has taken a 
flight on every airline in the world.” 


Solution: This statement is the negation of the statement “There is a woman who has taken a 
flight on every airline in the world” from Example 13. By Example 13, our statement can be 
expressed as ~dwVadf(P(w, f) A O(f, a)), where P(w, f) is “w has taken f” and QO(f, a) 
is “f is a flight on a.” By successively applying De Morgan’s laws for quantifiers in Table 2 
of Section 1.4 to move the negation inside successive quantifiers and by applying De Morgan’s 
law for negating a conjunction in the last step, we find that our statement is equivalent to each 
of this sequence of statements: 


Vw-Vai f (P(w, f) \ O(f, a)) = Vwia4 f (Pw, f) A O(f,a)) 
= VwiaV f-(P(w, f) A OC, a)) 
= Vwiav f (-P(w, f) V AO(f, a)). 


This last statement states “For every woman there is an airline such that for all flights, this 
woman has not taken that flight or that flight is not on this airline.” 
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(Requires calculus) Use quantifiers and predicates to express the fact that lim,—., f(x) does 
not exist where f(x) is a real-valued function of a real variable x and a belongs to the domain 


Solution: To say that lim,;.g f(x) does not exist means that for all real numbers L, 
lim, +¢ f(x) # L. By using Example 8, the statement lim,_., f(x) # L can be expressed as 


Successively applying the rules for negating quantified expressions, we construct this sequence 


In the last step we used the equivalence —(p > q) = p A -—q, which follows from the fifth 


Because the statement “lim,., f(x) does not exist” means for all real numbers L, 


This last statement says that for every real number L there is a real number € > O such that 
for every real number 6 > 0, there exists a real number x such that 0 < |x —a| <6 and 


EXAMPLE 16 
of f 
AWe > 055 >0Vx(0 < |x -—al <5 > |f(x)-—L| <e). 
of equivalent statements 
=AVe > 0 55 >0 Vx(0<|x —a| <6 > | f(x) — L| <e) 
= de>0 -56 >0 Vx(0<|x — al <d > | f(x) — L| <e) 
= de>0 Vb >0 -Vx(0<|x —a|<d > | f(x) — L| <e) 
= de>0V5>0 Ax -(0<|x —a|<6d > | f(x) -— L| <e) 
= de>0Vb>04x(0< |x —al<dA|f(x) —L|>e). 
equivalence in Table 7 of Section 1.3. 
lim,+a f(x) 4 L, this can be expressed as 
VLie >0 V5 >04x(0 < |x —al < dA |f(x) — L| > €). 
If@) —L| 2 €. 
Exercises 


1. Translate these statements into English, where the domain 


for each variable consists of all real numbers. 
a) Vxdy(x < y) 

b) VxVy(@ = 0) A (y = 0)) > (ry = 0)) 
c) VxVydz(xy = z) 


2. Translate these statements into English, where the domain 


for each variable consists of all real numbers. 
a) AxVy(xy = y) 

b) VxVy(( = 0) A (y < 0)) > @— y > 0)) 
c) WxVydz(x = y4+z) 


. Let Q(x, y) be the statement “x has sent an e-mail mes- 


sage to y,” where the domain for both x and y consists of 
all students in your class. Express each of these quantifi- 
cations in English. 


a) dxdyQ(x, y) 
c) VxdyQ(x, y) 
e) Vyax Q(x, y) 


b) AxVyQ(, y) 
d) Ay¥x Q(x, y) 
f) VxVyQ(x, y) 


. Let P(x, y) be the statement “Student x has taken class 


y,” where the domain for x consists of all students in your 
class and for y consists of all computer science courses 


at your school. Express each of these quantifications in 
English. 

a) dxdyP(x, y) 
ce) VxdyP(x, y) 
e) Vydx P(x, y) 


b) axVy P(x, y) 
d) AyVx P(x, y) 
f) VxVyP(x, y) 


. Let W(x, y) mean that student x has visited website y, 


where the domain for x consists of all students in your 

school and the domain for y consists of all websites. Ex- 

press each of these statements by a simple English sen- 

tence. 

a) W(Sarah Smith, www.att.com) 

b) 4x W(x, www.imdb.org) 

c) dyW(José Orez, y) 

d) Ay(W (Ashok Puri, y) A W(Cindy Yoon, y)) 

e) AyVz(y 4 (David Belcher) \ (W(David Belcher, z) 
> Wvy.2)) 

f) dxdyVz((x 4 y) AWG, 2) > Wy, 2))) 


. Let C(x, y) mean that student x is enrolled in class y, 


where the domain for x consists of all students in your 
school and the domain for y consists of all classes being 
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given at your school. Express each of these statements by 
a simple English sentence. 


a) C(Randy Goldberg, CS 252) 

b) 4xC(x, Math 695) 

c) dyC(Carol Sitea, y) 

d) Ax(C(x, Math 222) A C(x, CS 252)) 

e) AxdyVz((x 4 y) A (C(x, z) > CQ, 2))) 
f) dxdyVz((x # y) A (CG, 2) > Cy, 2) 


. Let T(x, y) mean that student x likes cuisine y, where the 


domain for x consists of all students at your school and 
the domain for y consists of all cuisines. Express each of 
these statements by a simple English sentence. 
a) —T (Abdallah Hussein, Japanese) 
b) AxT (x, Korean) A VxT (x, Mexican) 
c) dy(T(Monique Arsenault, y) Vv 
T (Jay Johnson, y)) 
d) VxVzAy(x #z) > A(T, y) ATE, y))) 
e) AxagVy(T(x, y) > T&, y)) 
f) VxVzdy(T (x, y) <> TC, y)) 


. Let Q(x, y) be the statement “student x has been a con- 


testant on quiz show y.” Express each of these sentences 
in terms of Q(x, y), quantifiers, and logical connectives, 
where the domain for x consists of all students at your 
school and for y consists of all quiz shows on television. 


a) There is a student at your school who has been a con- 
testant on a television quiz show. 

b) No student at your school has ever been a contestant 
on a television quiz show. 

c) There is a student at your school who has been a con- 
testant on Jeopardy and on Wheel of Fortune. 

d) Every television quiz show has had a student from 
your school as a contestant. 

e) Atleast two students from your school have been con- 
testants on Jeopardy. 


. Let L(x, y) be the statement “x loves y,” where the do- 


main for both x and y consists of all people in the world. 
Use quantifiers to express each of these statements. 


a) Everybody loves Jerry. 

b) Everybody loves somebody. 

c) There is somebody whom everybody loves. 

d) Nobody loves everybody. 

e) There is somebody whom Lydia does not love. 

f) There is somebody whom no one loves. 

g) There is exactly one person whom everybody loves. 

h) There are exactly two people whom Lynn loves. 

i) Everyone loves himself or herself. 

j) There is someone who loves no one besides himself 
or herself. 


Let F(x, y) be the statement “x can fool y,” where the 
domain consists of all people in the world. Use quantifiers 
to express each of these statements. 


a) Everybody can fool Fred. 

b) Evelyn can fool everybody. 

c) Everybody can fool somebody. 

d) There is no one who can fool everybody. 
e) Everyone can be fooled by somebody. 
f) No one can fool both Fred and Jerry. 

g) Nancy can fool exactly two people. 
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h) There is exactly one person whom everybody can fool. 

i) No one can fool himself or herself. 

j) There is someone who can fool exactly one person 

besides himself or herself. 

Let S(x) be the predicate “‘x is a student,’ F(x) the pred- 

icate “x is a faculty member,” and A(x, y) the predicate 

“x has asked y a question,” where the domain consists of 

all people associated with your school. Use quantifiers to 

express each of these statements. 

a) Lois has asked Professor Michaels a question. 

b) Every student has asked Professor Gross a question. 

c) Every faculty member has either asked Professor 

Miller a question or been asked a question by Pro- 
fessor Miller. 

d) Some student has not asked any faculty member a 

question. 

e) There is a faculty member who has never been asked 

a question by a student. 

f) Some student has asked every faculty member a ques- 

tion. 

g) There is a faculty member who has asked every other 

faculty member a question. 

h) Some student has never been asked a question by a 

faculty member. 

Let / (x) be the statement “x has an Internet connection” 

and C(x, y) be the statement “x and y have chatted over 

the Internet,’ where the domain for the variables x and y 

consists of all students in your class. Use quantifiers to 

express each of these statements. 

a) Jerry does not have an Internet connection. 

b) Rachel has not chatted over the Internet with Chelsea. 

c) Jan and Sharon have never chatted over the Internet. 

d) No one in the class has chatted with Bob. 

e) Sanjay has chatted with everyone except Joseph. 

f) Someone in your class does not have an Internet con- 
nection. 

g) Not everyone in your class has an Internet connec- 
tion. 

h) Exactly one student in your class has an Internet con- 
nection. 

i) Everyone except one student in your class has an In- 
ternet connection. 

j) Everyone in your class with an Internet connection 
has chatted over the Internet with at least one other 
student in your class. 

k) Someone in your class has an Internet connection but 
has not chatted with anyone else in your class. 

1) There are two students in your class who have not 
chatted with each other over the Internet. 

m) There is a student in your class who has chatted with 
everyone in your class over the Internet. 

n) There are at least two students in your class who have 
not chatted with the same person in your class. 

0) There are two students in the class who between them 
have chatted with everyone else in the class. 
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Let M(x, y) be “x has sent y an e-mail message” and 
T (x, y) be “x has telephoned y,” where the domain con- 
sists of all students in your class. Use quantifiers to ex- 
press each of these statements. (Assume that all e-mail 
messages that were sent are received, which is not the 
way things often work.) 

a) Chou has never sent an e-mail message to Koko. 

b) Arlene has never sent an e-mail message to or tele- 
phoned Sarah. 

c) José has never received an e-mail message from Deb- 
orah. 

d) Every student in your class has sent an e-mail mes- 
sage to Ken. 

e) No one in your class has telephoned Nina. 

f) Everyone in your class has either telephoned Avi or 
sent him an e-mail message. 

g) Thereisa student in your class who has sent everyone 
else in your class an e-mail message. 

h) There is someone in your class who has either sent an 
e-mail message or telephoned everyone else in your 
class. 

i) There are two different students in your class who 
have sent each other e-mail messages. 

j) There is a student who has sent himself or herself an 
e-mail message. 

k) There is a student in your class who has not received 
an e-mail message from anyone else in the class and 
who has not been called by any other student in the 
class. 

1) Every student in the class has either received an e- 
mail message or received a telephone call from an- 
other student in the class. 

m) There are at least two students in your class such that 
one student has sent the other e-mail and the second 
student has telephoned the first student. 

n) There are two different students in your class who 
between them have sent an e-mail message to or tele- 
phoned everyone else in the class. 

Use quantifiers and predicates with more than one vari- 

able to express these statements. 

a) There is a student in this class who can speak Hindi. 

b) Every student in this class plays some sport. 

c) Some student in this class has visited Alaska but has 
not visited Hawaii. 

d) All students in this class have learned at least one pro- 
gramming language. 

e) There is a student in this class who has taken ev- 
ery course offered by one of the departments in this 
school. 

f) Some student in this class grew up in the same town 
as exactly one other student in this class. 

g) Every student in this class has chatted with at least 
one other student in at least one chat group. 

Use quantifiers and predicates with more than one vari- 

able to express these statements. 

a) Every computer science student needs a course in dis- 
crete mathematics. 
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b) There is a student in this class who owns a personal 
computer. 

c) Every student in this class has taken at least one com- 
puter science course. 

d) There is a student in this class who has taken at least 
one course in computer science. 

e) Every student in this class has been in every building 
on campus. 

f) There is a student in this class who has been in every 
room of at least one building on campus. 

g) Every student in this class has been in at least one 
room of every building on campus. 


A discrete mathematics class contains 1 mathematics ma- 

jor who is a freshman, 12 mathematics majors who are 

sophomores, 15 computer science majors who are sopho- 

mores, 2 mathematics majors who are juniors, 2 computer 

science majors who are juniors, and 1 computer science 

major who is a senior. Express each of these statements in 

terms of quantifiers and then determine its truth value. 

a) There is a student in the class who is a junior. 

b) Every student in the class is a computer science major. 

c) There is a student in the class who is neither a math- 
ematics major nor a junior. 

d) Every student in the class is either a sophomore or a 
computer science major. 

e) There is a major such that there is a student in the class 
in every year of study with that major. 

Express each of these system specifications using predi- 

cates, quantifiers, and logical connectives, if necessary. 

a) Every user has access to exactly one mailbox. 

b) There is a process that continues to run during all error 
conditions only if the kernel is working correctly. 

c) All users on the campus network can access all web- 
sites whose url has a .edu extension. 


*d) There are exactly two systems that monitor every re- 
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mote server. 


Express each of these system specifications using predi- 
cates, quantifiers, and logical connectives, if necessary. 


a) At least one console must be accessible during every 
fault condition. 

b) The e-mail address of every user can be retrieved 
whenever the archive contains at least one message 
sent by every user on the system. 

c) For every security breach there is at least one mecha- 
nism that can detect that breach if and only if there is 
a process that has not been compromised. 

d) There are at least two paths connecting every two dis- 
tinct endpoints on the network. 

e) No one knows the password of every user on the sys- 
tem except for the system administrator, who knows 
all passwords.[ 

Express each of these statements using mathematical and 

logical operators, predicates, and quantifiers, where the 

domain consists of all integers. 

a) The sum of two negative integers is negative. 

b) The difference of two positive integers is not neces- 
sarily positive. 
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c) The sum of the squares of two integers is greater than 
or equal to the square of their sum. 

d) The absolute value of the product of two integers is 
the product of their absolute values. 

Express each of these statements using predicates, quan- 

tifiers, logical connectives, and mathematical operators 

where the domain consists of all integers. 


a) The product of two negative integers is positive. 

b) The average of two positive integers is positive. 

c) The difference of two negative integers is not neces- 
sarily negative. 

d) The absolute value of the sum of two integers does 
not exceed the sum of the absolute values of these 
integers. 


Use predicates, quantifiers, logical connectives, and 
mathematical operators to express the statement that ev- 
ery positive integer is the sum of the squares of four in- 
tegers. 

Use predicates, quantifiers, logical connectives, and 
mathematical operators to express the statement that there 
is a positive integer that is not the sum of three squares. 
Express each of these mathematical statements using 
predicates, quantifiers, logical connectives, and mathe- 
matical operators. 


a) The product of two negative real numbers is positive. 

b) The difference of a real number and itself is zero. 

c) Every positive real number has exactly two square 
roots. 

d) A negative real number does not have a square root 
that is a real number. 

Translate each of these nested quantifications into an En- 

glish statement that expresses a mathematical fact. The 

domain in each case consists of all real numbers. 

a) AxVy@a+y=y) 

b) WxVy(((x = 0) A (y < 0)) > @—y > 0)) 

ce) dxdy((x < 0) A (y < 0)) A — y > 0)) 

d) VxVy((x £0) A (y £0) > Gry £0) 

Translate each of these nested quantifications into an En- 

glish statement that expresses a mathematical fact. The 

domain in each case consists of all real numbers. 

a) ArVy(@y = y) 

b) WxVy(((x <0) A (y <0) > (xy > 0)) 

c) Axdy((x? > y) A@ < y)) 

d) VxVydz(x + y =z) 

Let Q(x, y) be the statement “x + y = x — y.” If the do- 

main for both variables consists of all integers, what are 

the truth values? 

a) QC, 1) 

ce) VyQ(1, y) 

e) AxdyQ, y) 

g) dyVx Q(x, y) 

i) WxVy Q(x, y) 

Determine the truth value of each of these statements if 

the domain for all variables consists of all integers. 

a) Wndm(n2 < m) b) AnVm(n < m?) 

c) Vndm(n +m = 0) d) AnVm(nm = m) 


b) Q(2, 0) 

d) Ax Q(x, 2) 
f) VxdyQ(, y) 
h) Vyax Q(x, y) 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 
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e) Andam(n2 + m? =5) f) dnam(n2 + m? = 6) 

g) dIndam(n+m=4An—m=1) 

h) sAndam(n+m=4An—m = 2) 

i) VWaVmsp(p = (m +n)/2) 

Determine the truth value of each of these statements if 
the domain of each variable consists of all real numbers. 
a) Vxdy (x2 =y) b) Vxdy(x = y?) 

c) AxVy(xy = 0) d) Axdy(xa+y Ay + x) 
e) Vx(x #40 > Ay(xy = 1)) 

f) axVy(y 40-> xy = 1) 

g) Vxdyat+y=1) 

h) Axdy(x + 2y =2A2x+4y =5) 

i) Vxdya+y=2A2x—-—y=1) 

j) VxVyde(z = (x + y)/2) 

Suppose the domain of the propositional function P(x, y) 
consists of pairs x and y, where x is 1, 2, or 3 and y is 
1, 2, or 3. Write out these propositions using disjunctions 
and conjunctions. 

a) VxVyP(x, y) b) Axdy P(x, y) 

c) AxVy P(x, y) d) Vysx P(x, y) 

Rewrite each of these statements so that negations ap- 
pear only within predicates (that is, so that no negation 
is outside a quantifier or an expression involving logical 
connectives). 

a) mdydx P(x, y) b) -Vxdy P(x, y) 

c) =Ay(Q(y) A Vx7R(Xx, y)) 

d) —dy(ax R(x, y) V VxS(x, y)) 

e) -dy(VxdzT (x, y, z) V AxVzU (x, y, Z)) 

Express the negations of each of these statements so that 
all negation symbols immediately precede predicates. 

a) VWxdyVzT (x, y, Z) 

b) Vxdy P(x, y) V VxdyQ(x, y) 

ce) Vxdy(P(x, y) AAzR(x, y, Z)) 

d) Wxdy(P(x, y) > Q(x, y)) 

Express the negations of each of these statements so that 
all negation symbols immediately precede predicates. 

a) AzVyVxT (x, y, Zz) 

b) AxdyP(x, y) AVxVyQ(x, y) 

ce) Axay(QQ@, y) = QU, x)) 

d) Vydxdz(T (x, y,z) V Q(x, y)) 

Rewrite each of these statements so that negations ap- 
pear only within predicates (that is, so that no negation 
is outside a quantifier or an expression involving logical 
connectives). 

a) “VxVyP(x, y) b) -Vydx P(x, y) 

c) “VyVx(P(x, y) V O(x, y)) 

d) 4(x3y5P(x, y) AVaxy Q(x, y)) 

e) -Vx(AyVzP(x, y, z) AAzVyP(x, y, Z)) 

Find a common domain for the variables x, y, and z 
for which the statement VxVy((x 4 y) > Vz((z = x) V 
(z = y))) is true and another domain for which it is false. 


Find a common domain for the variables x, y, z, 
and w for which the statement VxVyVzdw((w 4 x) A 
(w # y) A (w ¥ 2Z)) is true and another common domain 
for these variables for which it is false. 
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36. Express each of these statements using quantifiers. Then 
form the negation of the statement so that no negation is 
to the left of a quantifier. Next, express the negation in 
simple English. (Do not simply use the phrase “It is not 
the case that.”’) 

a) No one has lost more than one thousand dollars play- 


ing the lottery. 
b) There is a student in this class who has chatted with 


exactly one other student. 

c) No student in this class has sent e-mail to exactly two 
other students in this class. 

d) Some student has solved every exercise in this book. 

e) No student has solved at least one exercise in every 
section of this book. 


37. Express each of these statements using quantifiers. Then 
form the negation of the statement so that no negation is 
to the left of a quantifier. Next, express the negation in 
simple English. (Do not simply use the phrase “It is not 
the case that.”) 

a) Every student in this class has taken exactly two math- 


ematics classes at this school. 

b) Someone has visited every country in the world except 
Libya. 

c) Noone has climbed every mountain in the Himalayas. 

d) Every movie actor has either been in a movie with 
Kevin Bacon or has been in a movie with someone 
who has been in a movie with Kevin Bacon. 


38. Express the negations of these propositions using quan- 

tifiers, and in English. 

a) Every student in this class likes mathematics. 

b) There is a student in this class who has never seen a 
computer. 

c) There is a student in this class who has taken every 
mathematics course offered at this school. 

d) There is a student in this class who has been in at least 
one room of every building on campus. 


39. Find a counterexample, if possible, to these universally 
quantified statements, where the domain for all variables 
consists of all integers. 

a) WxVy(x2 = y2 > x =y) 
b) Vxdy(y* = x) 
ce) WxVy(xy > x) 

40. Find a counterexample, if possible, to these universally 
quantified statements, where the domain for all variables 
consists of all integers. 

a) Vxdy(x = 1/y) 
b) Vxdy(y2 — x < 100) 
©) WxVy(x? # y?) 

41. Use quantifiers to express the associative law for multi- 
plication of real numbers. 

42. Use quantifiers to express the distributive laws of multi- 
plication over addition for real numbers. 

43. Use quantifiers and logical connectives to express the fact 
that every linear polynomial (that is, polynomial of de- 
gree 1) with real coefficients and where the coefficient of 
x is nonzero, has exactly one real root. 

44. Use quantifiers and logical connectives to express the fact 
that a quadratic polynomial with real number coefficients 
has at most two real roots. 


45. Determine the truth value of the statement Vxdy(xy = 1) 
if the domain for the variables consists of 
a) the nonzero real numbers. 

b) the nonzero integers. 

c) the positive real numbers. 

46. Determine the truth value of the statement IxVy(x < y?) 
if the domain for the variables consists of 
a) the positive real numbers. 

b) the integers. 

c) the nonzero real numbers. 

47. Show that the two statements —4xVyP(x, y) and 
Vxdy— P(x, y), where both quantifiers over the first vari- 
able in P(x, y) have the same domain, and both quanti- 
fiers over the second variable in P(x, y) have the same 
domain, are logically equivalent. 

* 48. Show that Vx P(x) V Vx Q(x) and VxVy(P(x) Vv Q(y)), 
where all quantifiers have the same nonempty domain, 
are logically equivalent. (The new variable y is used to 
combine the quantifications correctly.) 

*49, a) Show that Vx P(x) A dx Q(x) is logically equivalent 
to Vxdy (P(x) A Q(y)), where all quantifiers have 
the same nonempty domain. 

b) Show that Vx P(x) V 4x Q(x) is equivalent to Vxdy 
(P(x) V Q(y)), where all quantifiers have the same 
nonempty domain. 

A statement is in prenex normal form (PNF) if and only if it 

is of the form 


Q1x1 Q2x2 +++ Onxg P(%1, X2,-.-5 Xk), 
where each Q;,i = 1, 2,...,k, is either the existential quan- 
tifier or the universal quantifier, and P(x, ..., x4) is a pred- 


icate involving no quantifiers. For example, dxVy(P(x, y) A 
Q(y)) is in prenex normal form, whereas 4x P(x) V Vx Q(x) 
is not (because the quantifiers do not all occur first). 

Every statement formed from propositional variables, 
predicates, T, and F using logical connectives and quan- 
tifiers is equivalent to a statement in prenex normal form. 
Exercise 51 asks for a proof of this fact. 


* 50. Put these statements in prenex normal form. [Hint: Use 
logical equivalence from Tables 6 and 7 in Section 1.3, 
Table 2 in Section 1.4, Example 19 in Section 1.4, 
Exercises 45 and 46 in Section 1.4, and Exercises 48 and 
49.] 

a) dx P(x) V Ax Q(x) V A, where A is a proposition not 
involving any quantifiers. 

b) —(Vx P(x) V Vx Q(x)) 

ce) dx P(x) > Ax Q(x) 


** 51. Show how to transform an arbitrary statement to a state- 


ment in prenex normal form that is equivalent to the given 
statement. (Note: A formal solution of this exercise re- 
quires use of structural induction, covered in Section 5.3.) 


*52. Express the quantification 4!x P(x), introduced in Sec- 
tion 1.4, using universal quantifications, existential quan- 
tifications, and logical operators. 
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Rules of Inference 


Introduction 


Later in this chapter we will study proofs. Proofs in mathematics are valid arguments that estab- 
lish the truth of mathematical statements. By an argument, we mean a sequence of statements 
that end with a conclusion. By valid, we mean that the conclusion, or final statement of the 
argument, must follow from the truth of the preceding statements, or premises, of the argument. 
That is, an argument is valid if and only if it is impossible for all the premises to be true and 
the conclusion to be false. To deduce new statements from statements we already have, we use 
rules of inference which are templates for constructing valid arguments. Rules of inference are 
our basic tools for establishing the truth of statements. 

Before we study mathematical proofs, we will look at arguments that involve only compound 
propositions. We will define what it means for an argument involving compound propositions to 
be valid. Then we will introduce a collection of rules of inference in propositional logic. These 
rules of inference are among the most important ingredients in producing valid arguments. After 
we illustrate how rules of inference are used to produce valid arguments, we will describe some 
common forms of incorrect reasoning, called fallacies, which lead to invalid arguments. 

After studying rules of inference in propositional logic, we will introduce rules of inference 
for quantified statements. We will describe how these rules of inference can be used to produce 
valid arguments. These rules of inference for statements involving existential and universal 
quantifiers play an important role in proofs in computer science and mathematics, although they 
are often used without being explicitly mentioned. 

Finally, we will show how rules of inference for propositions and for quantified statements 
can be combined. These combinations of rule of inference are often used together in complicated 
arguments. 


Valid Arguments in Propositional Logic 


Consider the following argument involving propositions (which, by definition, is a sequence of 
propositions): 


“If you have a current password, then you can log onto the network.” 
“You have a current password.” 

Therefore, 

“You can log onto the network.” 


We would like to determine whether this is a valid argument. That is, we would like to 
determine whether the conclusion “You can log onto the network” must be true when the 
premises “If you have a current password, then you can log onto the network” and “You have a 
current password” are both true. 
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DEFINITION 1 


Before we discuss the validity of this particular argument, we will look at its form. Use p 
to represent “You have a current password” and q to represent “You can log onto the network.” 
Then, the argument has the form 


pq 
P 
|G 


where .*. is the symbol that denotes “therefore.” 

We know that when p and q are propositional variables, the statement ((p > qg) A p) > q 
is a tautology (see Exercise 10(c) in Section 1.3). In particular, when both p — gq and p are 
true, we know that g must also be true. We say this form of argument is valid because whenever 
all its premises (all statements in the argument other than the final one, the conclusion) are true, 
the conclusion must also be true. Now suppose that both “If you have a current password, then 
you can log onto the network” and “You have a current password” are true statements. When 
we replace p by “You have a current password” and qg by “You can log onto the network,” it 
necessarily follows that the conclusion “You can log onto the network” is true. This argument 
is valid because its form is valid. Note that whenever we replace p and q by propositions where 
p — q and p are both true, then g must also be true. 

What happens when we replace p and gq in this argument form by propositions where not 
both p and p — q are true? For example, suppose that p represents “You have access to the 
network” and g represents “You can change your grade” and that p is true, but p — q is false. 
The argument we obtain by substituting these values of p and q into the argument form is 


“Tf you have access to the network, then you can change your grade.” 
“You have access to the network.” 


.. “You can change your grade.” 


The argument we obtained is a valid argument, but because one of the premises, namely the first 
premise, is false, we cannot conclude that the conclusion is true. (Most likely, this conclusion 
is false.) 

In our discussion, to analyze an argument, we replaced propositions by propositional vari- 
ables. This changed an argument to an argument form. We saw that the validity of an argument 
follows from the validity of the form of the argument. We summarize the terminology used to 
discuss the validity of arguments with our definition of the key notions. 


An argument in propositional logic is a sequence of propositions. All but the final proposition 
in the argument are called premises and the final proposition is called the conclusion. An 
argument is valid if the truth of all its premises implies that the conclusion is true. 

An argument form in propositional logic is a sequence of compound propositions involv- 
ing propositional variables. An argument form is valid no matter which particular proposi- 
tions are substituted for the propositional variables in its premises, the conclusion is true if 
the premises are all true. 


From the definition of a valid argument form we see that the argument form with premises 
P1; P2,---+, Pn and conclusion gq is valid, when (p1 A p2 A--: A pn) > q is a tautology. 

The key to showing that an argument in propositional logic is valid is to show that its 
argument form is valid. Consequently, we would like techniques to show that argument forms 
are valid. We will now develop methods for accomplishing this task. 


EXAMPLE 1 


EXAMPLE 2 
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Rules of Inference for Propositional Logic 


We can always use a truth table to show that an argument form is valid. We do this by showing 
that whenever the premises are true, the conclusion must also be true. However, this can be 
a tedious approach. For example, when an argument form involves 10 different propositional 
variables, to use a truth table to show this argument form is valid requires 2!° = 1024 different 
rows. Fortunately, we do not have to resort to truth tables. Instead, we can first establish the 
validity of some relatively simple argument forms, called rules of inference. These rules of 
inference can be used as building blocks to construct more complicated valid argument forms. 
We will now introduce the most important rules of inference in propositional logic. 

The tautology (p A (p > q)) — q is the basis of the rule of inference called modus po- 
nens, or the law of detachment. (Modus ponens is Latin for mode that affirms.) This tautology 
leads to the following valid argument form, which we have already seen in our initial discussion 
about arguments (where, as before, the symbol .°. denotes “therefore’’): 


Using this notation, the hypotheses are written in a column, followed by a horizontal bar, followed 
by a line that begins with the therefore symbol and ends with the conclusion. In particular, modus 
ponens tells us that if a conditional statement and the hypothesis of this conditional statement 
are both true, then the conclusion must also be true. Example | illustrates the use of modus 
ponens. 


Suppose that the conditional statement “If it snows today, then we will go skiing” and its 
hypothesis, “It is snowing today,” are true. Then, by modus ponens, it follows that the conclusion 
of the conditional statement, “We will go skiing,” is true. < 


As we mentioned earlier, a valid argument can lead to an incorrect conclusion if one or 
more of its premises is false. We illustrate this again in Example 2. 


Determine whether the argument given here is valid and determine whether its conclusion must 
be true because of the validity of the argument. 


“Tf./2 > 3, then (/2)° > (3). We know that /2 > 3. Consequently, 


(v2) =2 > (3) = 3" 


Solution: Let p be the proposition “/2 > 3” and q the proposition “2 > (3)?” The premises 


of the argument are p — q and p, and q is its conclusion. This argument is valid because it 
is constructed by using modus ponens, a valid argument form. However, one of its premises, 


J/2> 3, is false. Consequently, we cannot conclude that the conclusion is true. Furthermore, 


note that the conclusion of this argument is false, because 2 < z < 


There are many useful rules of inference for propositional logic. Perhaps the most widely 
used of these are listed in Table 1. Exercises 9, 10, 15, and 30 in Section 1.3 ask for the 
verifications that these rules of inference are valid argument forms. We now give examples of 
arguments that use these rules of inference. In each argument, we first use propositional variables 
to express the propositions in the argument. We then show that the resulting argument form is 
a rule of inference from Table 1. 
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TABLE 1 Rules of Inference. 


oe 
(-q \(p > q)) > 7p Modus tollens 


(prQgAq>r))7- pr) Hypothetical syllogism 


(pVq)An7p)>4q Disjunctive syllogism 
[be —— 
' : " 7 : ae 


a 
a 


EXAMPLE 3 | State which rule of inference is the basis of the following argument: “It is below freezing now. 
Therefore, it is either below freezing or raining now.” 


Solution: Let p be the proposition “It is below freezing now” and q the proposition “It is raining 
now.” Then this argument is of the form 


P 
“PV 


This is an argument that uses the addition rule. < 


EXAMPLE 4 | State which rule of inference is the basis of the following argument: “It is below freezing and 
raining now. Therefore, it is below freezing now.” 


Solution: Let p be the proposition “It is below freezing now,” and let g be the proposition “It is 
raining now.” This argument is of the form 


DAW 
D 


This argument uses the simplification rule. < 


EXAMPLE 5 


EXAMPLE 6 


Extra 
Examples Qing 
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State which rule of inference is used in the argument: 


If it rains today, then we will not have a barbecue today. If we do not have a barbecue today, 
then we will have a barbecue tomorrow. Therefore, if it rains today, then we will have a 
barbecue tomorrow. 


Solution: Let p be the proposition “It is raining today,” let g be the proposition “We will not 
have a barbecue today,’ and let r be the proposition “We will have a barbecue tomorrow.” Then 
this argument is of the form 


pq 
q->r 


por 


Hence, this argument is a hypothetical syllogism. < 


Using Rules of Inference to Build Arguments 


When there are many premises, several rules of inference are often needed to show that an 
argument is valid. This is illustrated by Examples 6 and 7, where the steps of arguments are 
displayed on separate lines, with the reason for each step explicitly stated. These examples also 
show how arguments in English can be analyzed using rules of inference. 


Show that the premises “It is not sunny this afternoon and it is colder than yesterday,” “We will 
go swimming only if it is sunny,” “If we do not go swimming, then we will take a canoe trip,” 
and “If we take a canoe trip, then we will be home by sunset” lead to the conclusion “We will 
be home by sunset.” 


Solution: Let p be the proposition “It is sunny this afternoon,” g the proposition “It is colder 
than yesterday,” r the proposition “We will go swimming,” s the proposition “We will take a 
canoe trip,” and ¢ the proposition “We will be home by sunset.” Then the premises become 
=p Aq, > p,-7r — s, and s > t. The conclusion is simply t. We need to give a valid 
argument with premises =p A q,r — p, -r — s, ands — t¢ and conclusion ft. 

We construct an argument to show that our premises lead to the desired conclusion as 
follows. 


Step Reason 

l. =pAq Premise 

2. 4p Simplification using (1) 
3.r>p Premise 

4. =r Modus tollens using (2) and (3) 
5. -r > s Premise 

6. s Modus ponens using (4) and (5) 
7T.s—>t Premise 

8. t Modus ponens using (6) and (7) 


Note that we could have used a truth table to show that whenever each of the four hypotheses 
is true, the conclusion is also true. However, because we are working with five propositional 
variables, p, g, r,s, and ft, such a truth table would have 32 rows. 
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EXAMPLE 7 Show that the premises “If you send me an e-mail message, then I will finish writing the 


Links 


EXAMPLE 8 


Extra & 
Examples Qi 


program,” “If you do not send me an e-mail message, then I will go to sleep early,” and “If I go 
to sleep early, then I will wake up feeling refreshed” lead to the conclusion “If I do not finish 
writing the program, then I will wake up feeling refreshed.” 


Solution: Let p be the proposition “You send me an e-mail message,” g the proposition “T will 
finish writing the program,” r the proposition “I will go to sleep early,” and s the proposition “I 
will wake up feeling refreshed.” Then the premises are p > g, =p > r,andr — s.The desired 
conclusion is ~g — s. We need to give a valid argument with premises p > g, =p —> r, and 
r — sand conclusion =q —> s. 

This argument form shows that the premises lead to the desired conclusion. 


Step Reason 
l. pq Premise 
2. -q > 7p Contrapositive of (1) 
3. apor Premise 
4. -q->r Hypothetical syllogism using (2) and (3) 
5.r>as Premise 4 
6. -q-> 5s Hypothetical syllogism using (4) and (5) 
Resolution 


Computer programs have been developed to automate the task of reasoning and proving theo- 
rems. Many of these programs make use of a rule of inference known as resolution. This rule 
of inference is based on the tautology 


(pV q)A\ (pvr) @vr). 


(Exercise 30 in Section 1.3 asks for the verification that this is a tautology.) The final disjunction in 
the resolution rule, g V r, is called the resolvent. When we let g = r in this tautology, we obtain 
(pV q) A (-=p Vv gq) > q. Furthermore, when we let r = F, we obtain (p V q) A (=p) > q 
(because g V F = q), whichis the tautology on which the rule of disjunctive syllogism is based. 


Use resolution to show that the hypotheses “Jasmine is skiing or it is not snowing” and “‘It is 
snowing or Bart is playing hockey” imply that “Jasmine is skiing or Bart is playing hockey.” 


Solution: Let p be the proposition “It is snowing,” g the proposition “Jasmine is skiing,” and r 
the proposition “Bart is playing hockey.” We can represent the hypotheses as =p V g and p V r, 
respectively. Using resolution, the proposition g Vr, “Jasmine is skiing or Bart is playing 
hockey,” follows. 


Resolution plays an important role in programming languages based on the rules of logic, 
such as Prolog (where resolution rules for quantified statements are applied). Furthermore, it 
can be used to build automatic theorem proving systems. To construct proofs in propositional 
logic using resolution as the only rule of inference, the hypotheses and the conclusion must be 
expressed as clauses, where a clause is a disjunction of variables or negations of these variables. 
We can replace a statement in propositional logic that is not a clause by one or more equivalent 
statements that are clauses. For example, suppose we have a statement of the form p Vv (¢ Ar). 
Because p V (¢ Ar) = (pV q)A (pV r), we can replace the single statement p V (¢ Ar) by 
two statements p V gq and p Vr, each of which is a clause. We can replace a statement of 
the form —(p Vv q) by the two statements —p and —q because De Morgan’s law tells us that 
=(p V q) = =p A 7g. We can also replace a conditional statement p — q with the equivalent 
disjunction =p V q. 


EXAMPLE 9 


Links 


EXAMPLE 10 


EXAMPLE 11 
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Show that the premises (p A g) V r andr — s imply the conclusion p V s. 


Solution: We can rewrite the premises (p A q) V r as two clauses, p V r andg V r. We can also 
replace r — s by the equivalent clause sr V s. Using the two clauses p V r and =r V s, we can 
use resolution to conclude p V s. 


Fallacies 


Several common fallacies arise in incorrect arguments. These fallacies resemble rules of infer- 
ence, but are based on contingencies rather than tautologies. These are discussed here to show 
the distinction between correct and incorrect reasoning. 

The proposition ((p > q) Aq) —> p 1s not a tautology, because it is false when p is false 
and q is true. However, there are many incorrect arguments that treat this as a tautology. In 
other words, they treat the argument with premises p — q and qg and conclusion p as a valid 
argument form, which it is not. This type of incorrect reasoning is called the fallacy of affirming 
the conclusion. 


Is the following argument valid? 


If you do every problem in this book, then you will learn discrete mathematics. You learned 
discrete mathematics. 


Therefore, you did every problem in this book. 


Solution: Let p be the proposition “You did every problem in this book.” Let g be the proposition 
“You learned discrete mathematics.” Then this argument is of the form: if p — q and q, then 
p. This is an example of an incorrect argument using the fallacy of affirming the conclusion. 
Indeed, it is possible for you to learn discrete mathematics in some way other than by doing every 
problem in this book. (You may learn discrete mathematics by reading, listening to lectures, 
doing some, but not all, the problems in this book, and so on.) < 


The proposition ((p > q) A =p) — 7¢ is not a tautology, because it is false when p is 
false and q is true. Many incorrect arguments use this incorrectly as a rule of inference. This 
type of incorrect reasoning is called the fallacy of denying the hypothesis. 


Let p and qg be as in Example 10. If the conditional statement p — q is true, and —p is true, 
is it correct to conclude that —g is true? In other words, is it correct to assume that you did not 
learn discrete mathematics if you did not do every problem in the book, assuming that if you do 
every problem in this book, then you will learn discrete mathematics? 


Solution: It 1s possible that you learned discrete mathematics even if you did not do every 
problem in this book. This incorrect argument is of the form p — qg and —p imply —q, which 
is an example of the fallacy of denying the hypothesis. 


Rules of Inference for Quantified Statements 


We have discussed rules of inference for propositions. We will now describe some important rules 
of inference for statements involving quantifiers. These rules of inference are used extensively 
in mathematical arguments, often without being explicitly mentioned. 

Universal instantiation is the rule of inference used to conclude that P(c) is true, where c 
is a particular member of the domain, given the premise Vx P (x). Universal instantiation is used 
when we conclude from the statement “All women are wise” that “Lisa is wise,” where Lisa is 
a member of the domain of all women. 
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EXAMPLE 12 
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TABLE 2 Rules of Inference for Quantified Statements. 


Vx P ‘ : ae 
sea Universal instantiation 
.. P(c) 


P(c) for an arbitrary c 
-. Vx P(x) 


Universal generalization 


dx P(x) 


Existential instantiation 
.. P(c) for some element c 


P(c) for some element c 
J. dx P(x) 


Existential generalization 


Universal generalization is the rule of inference that states that Vx P(x) is true, given the 
premise that P (c) is true for all elements c in the domain. Universal generalization is used when 
we show that Vx P (x) is true by taking an arbitrary element c from the domain and showing that 
P(c) is true. The element c that we select must be an arbitrary, and not a specific, element of 
the domain. That is, when we assert from Vx P(x) the existence of an element c in the domain, 
we have no control over c and cannot make any other assumptions about c other than it comes 
from the domain. Universal generalization is used implicitly in many proofs in mathematics and 
is seldom mentioned explicitly. However, the error of adding unwarranted assumptions about 
the arbitrary element c when universal generalization is used is all too common in incorrect 
reasoning. 

Existential instantiation is the rule that allows us to conclude that there is an element c in 
the domain for which P(c) is true if we know that 4x P(x) is true. We cannot select an arbitrary 
value of c here, but rather it must be a c for which P(c) is true. Usually we have no knowledge 
of what c is, only that it exists. Because it exists, we may give it a name (c) and continue our 
argument. 

Existential generalization is the rule of inference that is used to conclude that 4x P(x) is 
true when a particular element c with P(c) true is known. That is, if we know one element c in 
the domain for which P(c) is true, then we know that 4x P(x) is true. 

We summarize these rules of inference in Table 2. We will illustrate how some of these rules 
of inference for quantified statements are used in Examples 12 and 13. 


Show that the premises “Everyone in this discrete mathematics class has taken a course in 
computer science” and ‘“‘Marla is a student in this class” imply the conclusion “Marla has taken 
a course in computer science.” 


Solution: Let D(x) denote “‘x is in this discrete mathematics class,” and let C(x) denote “x has 
taken a course in computer science.” Then the premises are Vx (D(x) — C(x)) and D(Marla). 
The conclusion is C(Marla). 

The following steps can be used to establish the conclusion from the premises. 


Step Reason 
1. Vx(D(x) > C(x)) Premise 
2. D(Marla) — C(Marla) Universal instantiation from (1) 
3. D(Marla) Premise 


4. C(Marla) Modus ponens from (2) and (3) < 


EXAMPLE 13 


EXAMPLE 14 
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Show that the premises “A student in this class has not read the book,” and “Everyone in this 
class passed the first exam” imply the conclusion “Someone who passed the first exam has not 
read the book.” 


Solution: Let C(x) be “x is in this class,” B(x) be “x has read the book,” and P(x) be “x passed 
the first exam.” The premises are 4x(C(x) A =B(x)) and Vx(C(x) > P(x)). The conclusion 
is dx(P(x) A —B(x)). These steps can be used to establish the conclusion from the premises. 


Step Reason 

1. dx(C(x) A =B(x)) Premise 

2. C(a) A =B(a) Existential instantiation from (1) 

3. C(a) Simplification from (2) 

4. Vx(C(x) > P(x)) Premise 

5. C(a) > P(a) Universal instantiation from (4) 

6. P(a) Modus ponens from (3) and (5) 

7. =B(a) Simplification from (2) 

8. P(a) A =B(a) Conjunction from (6) and (7) 

9. Ax(P(x) A =B(x)) Existential generalization from (8) 4 


Combining Rules of Inference for Propositions 
and Quantified Statements 


We have developed rules of inference both for propositions and for quantified statements. Note 
that in our arguments in Examples 12 and 13 we used both universal instantiation, a rule of 
inference for quantified statements, and modus ponens, arule of inference for propositional logic. 
We will often need to use this combination of rules of inference. Because universal instantiation 
and modus ponens are used so often together, this combination of rules is sometimes called 
universal modus ponens. This rule tells us that if Vx(P(x) — Q(x)) is true, and if P(a) is 
true for a particular element a in the domain of the universal quantifier, then Q(a) must also 
be true. To see this, note that by universal instantiation, P(a) > Q(a) is true. Then, by modus 
ponens, Q(a) must also be true. We can describe universal modus ponens as follows: 


Wx(P(x) > Q(x)) 


P(a), where a is a particular element in the domain 


“. O(a) 


Universal modus ponens is commonly used in mathematical arguments. This is illustrated 
in Example 14. 


Assume that “For all positive integers n, if n is greater than 4, then n” is less than 2”” is true. 
Use universal modus ponens to show that 1007 < 2!, 


Solution: Let P(n) denote “n > 4” and Q(n) denote “n* < 2”.” The statement “For all positive 
integers n, ifn is greater than 4, then n7 is less than 2”” canbe represented by Vn(P(n) > Q(n)), 
where the domain consists of all positive integers. We are assuming that Vn(P(n) > Q(n)) is 
true. Note that P(100) is true because 100 > 4. It follows by universal modus ponens that 
Q(100) is true, namely that 1002 < 2100, 


Another useful combination of a rule of inference from propositional logic and a rule 
of inference for quantified statements is universal modus tollens. Universal modus tollens 
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combines universal instantiation and modus tollens and can be expressed in the following way: 


Wx(P(x) > Q(x)) 


=Q(a), where a isa particular element in the domain 


*--P(a) 


The verification of universal modus tollens is left as Exercise 25. Exercises 26-29 develop 
additional combinations of rules of inference in propositional logic and quantified statements. 


Exercises 


1. Find the argument form for the following argument and 


determine whether it is valid. Can we conclude that the 
conclusion is true if the premises are true? 


If Socrates is human, then Socrates is mortal. 
Socrates is human. 


.. Socrates is mortal. 


. Find the argument form for the following argument and 


determine whether it is valid. Can we conclude that the 
conclusion is true if the premises are true? 


If George does not have eight legs, then he is not a 
spider. 
George is a spider. 


‘, George has eight legs. 


3. What rule of inference is used in each of these argu- 


ments? 

a) Alice is a mathematics major. Therefore, Alice is ei- 
ther a mathematics major or a computer science major. 

b) Jerry is a mathematics major and a computer science 
major. Therefore, Jerry is a mathematics major. 

c) If itis rainy, then the pool will be closed. It is rainy. 
Therefore, the pool is closed. 

d) If it snows today, the university will close. The uni- 
versity is not closed today. Therefore, it did not snow 
today. 

e) IfI go swimming, then I will stay in the sun too long. 
If I stay in the sun too long, then I will sunburn. There- 
fore, if I go swimming, then I will sunburn. 


4. What rule of inference is used in each of these arguments? 


a) Kangaroos live in Australia and are marsupials. There- 
fore, kangaroos are marsupials. 

b) It is either hotter than 100 degrees today or the pollu- 
tion is dangerous. It is less than 100 degrees outside 
today. Therefore, the pollution is dangerous. 

c) Lindais an excellent swimmer. If Linda is an excellent 
swimmer, then she can work as a lifeguard. Therefore, 
Linda can work as a lifeguard. 

d) Steve will work at a computer company this summer. 
Therefore, this summer Steve will work at a computer 
company or he will be a beach bum. 


e) If I work all night on this homework, then I can an- 
swer all the exercises. If I answer all the exercises, I 
will understand the material. Therefore, if I work all 
night on this homework, then I will understand the 
material. 


. Userules of inference to show that the hypotheses “Randy 


works hard,” “If Randy works hard, then he is a dull boy,” 
and “If Randy is a dull boy, then he will not get the job” 
imply the conclusion “Randy will not get the job.” 


. Use rules of inference to show that the hypotheses “If it 


does not rain or if it is not foggy, then the sailing race will 
be held and the lifesaving demonstration will go on,” “If 
the sailing race is held, then the trophy will be awarded,” 
and “The trophy was not awarded” imply the conclusion 
“Tt rained.” 


. What rules of inference are used in this famous argu- 


ment? “All men are mortal. Socrates is a man. Therefore, 
Socrates is mortal.” 


. What rules of inference are used in this argument? “No 


man is an island. Manhattan is an island. Therefore, Man- 
hattan is not a man.” 


. For each of these collections of premises, what relevant 


conclusion or conclusions can be drawn? Explain the 
tules of inference used to obtain each conclusion from 
the premises. 


a) “If I take the day off, it either rains or snows.” “I took 
Tuesday off or I took Thursday off.” “It was sunny on 
Tuesday.” “It did not snow on Thursday.” 

b) “If I eat spicy foods, then I have strange dreams.” “I 
have strange dreams if there is thunder while I sleep.” 
“T did not have strange dreams.” 

c) “Iam either clever or lucky.” “I am not lucky.” “If I 
am lucky, then I will win the lottery.” 

d) “Every computer science major has a personal com- 
puter.” “Ralph does not have a personal computer.” 
“Ann has a personal computer.” 

e) “What is good for corporations is good for the United 
States.” “What is good for the United States is good 
for you.” “What is good for corporations is for you to 
buy lots of stuff.” 

f) “All rodents gnaw their food.” “Mice are rodents.” 
“Rabbits do not gnaw their food.” “Bats are not ro- 
dents.” 


10. For each of these sets of premises, what relevant conclu- 


11 


12 


13 


14 


sion or conclusions can be drawn? Explain the rules of in- 
ference used to obtain each conclusion from the premises. 


a) “If I play hockey, then I am sore the next day.” “I 
use the whirlpool if I am sore.” “I did not use the 
whirlpool.” 


b) “IfI work, itis either sunny or partly sunny.” “I worked 
last Monday or I worked last Friday.” “It was not sunny 
on Tuesday.” “It was not partly sunny on Friday.” 


c) “All insects have six legs.” “Dragonflies are insects.” 
“Spiders do not have six legs.” “Spiders eat dragon- 
flies.” 


d) “Every student has an Internet account.” “Homer does 
not have an Internet account.” “Maggie has an Internet 
account.” 


e) “All foods that are healthy to eat do not taste good.” 
“Tofu is healthy to eat.’ “You only eat what tastes 
good.” “You do not eat tofu.” “Cheeseburgers are not 
healthy to eat.” 


f) “I am either dreaming or hallucinating.” “I am not 
dreaming.” “If I am hallucinating, I see elephants run- 
ning down the road.” 


Show that the argument form with premises 
P1; P2:+++;Pn and conclusion g — r is valid if the 
argument form with premises pj, p2,..., Pn,g, and 
conclusion r is valid. 


Show that the argument form with premises (p A t) > 
(rVs), gq —> UAt), u— p, and —s and conclusion 
q — r is valid by first using Exercise 11 and then us- 
ing rules of inference from Table 1. 


For each of these arguments, explain which rules of in- 
ference are used for each step. 


a) “Doug, a student in this class, knows how to write 
programs in JAVA. Everyone who knows how to write 
programs in JAVA can get a high-paying job. There- 
fore, someone in this class can get a high-paying job.” 


b) “Somebody in this class enjoys whale watching. Ev- 
ery person who enjoys whale watching cares about 
ocean pollution. Therefore, there is a person in this 
class who cares about ocean pollution.” 


c) “Each of the 93 students in this class owns a personal 
computer. Everyone who owns a personal computer 
can use a word processing program. Therefore, Zeke, 
a student in this class, can use a word processing pro- 
gram.” 


d) “Everyone in New Jersey lives within 50 miles of the 
ocean. Someone in New Jersey has never seen the 
ocean. Therefore, someone who lives within 50 miles 
of the ocean has never seen the ocean.” 

For each of these arguments, explain which rules of in- 

ference are used for each step. 

a) “Linda, a student in this class, owns a red convertible. 
Everyone who owns a red convertible has gotten at 
least one speeding ticket. Therefore, someone in this 
class has gotten a speeding ticket.” 
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b) “Each of five roommates, Melissa, Aaron, Ralph, Ve- 
neesha, and Keeshawn, has taken a course in discrete 
mathematics. Every student who has taken a course in 
discrete mathematics can take a course in algorithms. 
Therefore, all five roommates can take a course in 
algorithms next year.” 

c) “All movies produced by John Sayles are wonder- 
ful. John Sayles produced a movie about coal miners. 
Therefore, there is a wonderful movie about coal min- 
ers.” 

d) “There is someone in this class who has been to 
France. Everyone who goes to France visits the 
Louvre. Therefore, someone in this class has visited 
the Louvre.” 


For each of these arguments determine whether the argu- 

ment is correct or incorrect and explain why. 

a) All students in this class understand logic. Xavier is 
a student in this class. Therefore, Xavier understands 
logic. 

b) Every computer science major takes discrete math- 
ematics. Natasha is taking discrete mathematics. 
Therefore, Natasha is a computer science major. 

c) All parrots like fruit. My pet bird is not a parrot. There- 
fore, my pet bird does not like fruit. 

d) Everyone who eats granola every day is healthy. Linda 
is not healthy. Therefore, Linda does not eat granola 
every day. 

For each of these arguments determine whether the argu- 

ment is correct or incorrect and explain why. 


a) Everyone enrolled in the university has lived in a dor- 
mitory. Mia has never lived in a dormitory. Therefore, 
Mia is not enrolled in the university. 

b) A convertible car is fun to drive. Isaac’s car is not a 
convertible. Therefore, Isaac’s car is not fun to drive. 

c) Quincy likes all action movies. Quincy likes the movie 
Eight Men Out. Therefore, Eight Men Out is an action 
movie. 

d) All lobstermen set at least a dozen traps. Hamilton is a 
lobsterman. Therefore, Hamilton sets at least a dozen 
traps. 


What is wrong with this argument? Let H(x) be “x is 
happy.” Given the premise 4x H(x), we conclude that 
H (Lola). Therefore, Lola is happy. 


What is wrong with this argument? Let S(x, y) be “x is 
shorter than y.” Given the premise 4s'S(s, Max), it follows 
that S(Max, Max). Then by existential generalization it 
follows that 4x S(x, x), so that someone is shorter than 
himself. 


Determine whether each of these arguments is valid. If an 
argument is correct, what rule of inference is being used? 
If it is not, what logical error occurs? 


a) If n is a real number such that n > 1, then n2 > 1. 
Suppose that n? > 1. Thenn > 1. 

b) If n is a real number with n > 3, then n2 > 9, 
Suppose that n2 < 9. Thenn < 3. 

c) If n is a real number with n > 2, then n2 > 4, 
Suppose that n < 2. Then n2 < 4. 
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Determine whether these are valid arguments. 
a) Ifx isa positive real ee then x? is a positive real 
number. Therefore, if a? is positive, where a is a real 


number, then a is a positive real number. 
b) If x2 ¢ 0, where x isa Be number, then x 4 0. Let 


a be areal number with a? £ 0; then a # 0. 

Which rules of inference are used to establish the 
conclusion of Lewis Carroll’s argument described in 
Example 26 of Section 1.4? 

Which rules of inference are used to establish the 
conclusion of Lewis Carroll’s argument described in 
Example 27 of Section 1.4? 

Identify the error or errors in this argument that sup- 
posedly shows that if 4x P(x) A 4x Q(x) is true then 
dx(P(x) A Q(x)) is true. 


1. ax P(x) VAxQ(x) Premise 


2. dx P(x) Simplification from (1) 

3. P(c) Existential instantiation from (2) 
4. 4x Q(x) Simplification from (1) 

5. Q(c) Existential instantiation from (4) 
6. P(c) A Q(c) Conjunction from (3) and (5) 

7. Ax(P(x) A Q(x)) — Existential generalization 
Identify the error or errors in this argument that sup- 


posedly shows that if Vx(P(x) V Q(x)) is true then 
Vx P(x) V Vx Q(x) is true. 
1. Vx(P(x) V Q(x)) 


Premise 


2. P(c) V O(c) Universal instantiation from (1) 
3. P(c) Simplification from (2) 

4. Vx P(x) Universal generalization from (3) 
5. Q(c) Simplification from (2) 

6. Vx O(x) Universal generalization from (5) 
7. Vx(P (x) Vv Vx Q(x)) Conjunction from (4) and (6) 
Justify the rule of universal modus tollens by showing 


that the premises Vx(P(x) > Q(x)) and —Q(a) for a 
particular element a in the domain, imply —P(a). 
Justify the rule of universal transitivity, which states that 
if Vx(P(x) > Q(x)) and Vx(Q(x) > R(x)) are true, 
then Vx(P(x) — R(x)) is true, where the domains of all 
quantifiers are the same. 

Use rules of inference to show that if Vx(P(x) > 
(Q(x) A S(x))) and Vx(P(x) A R(x)) are true, then 
Vx(R(x) A S(x)) is true. 

Use rules of inference to show that if Vx(P(x) Vv 
Q(x)) and Vx((—=P(x) A Q(x)) > R(x)) are true, then 
Vx(-R(x) > P(x)) is also true, where the domains of 
all quantifiers are the same. 
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Use rules of inference to show that if Vx(P(x) V Q(x)), 
Vx(-Q(x) Vv S(x)), Vx(R(x) > -S(x)), and 4ax-P (x) 
are true, then dx—R(x) is true. 


Use resolution to show the hypotheses “Allen is a bad 
boy or Hillary is a good girl” and “Allen is a good boy or 
David is happy” imply the conclusion “Hillary is a good 
girl or David is happy.” 


Use resolution to show that the hypotheses “It is not rain- 
ing or Yvette has her umbrella,” “Yvette does not have 
her umbrella or she does not get wet,” and “It is raining 
or Yvette does not get wet” imply that “Yvette does not 
get wet.” 


Show that the equivalence p A ~p = F can be derived 
using resolution together with the fact that a condi- 
tional statement with a false hypothesis is true. [Hint: Let 
g =r = F in resolution.] 


Use resolution to show that the compound propo- 
sition (p V q) A (=p V q) A (pV 7g) A (=p V 79) is 
not satisfiable. 
The Logic Problem, taken from WFF’N PROOF, The 
Game of Logic, has these two assumptions: 
I. “Logic is difficult or not many students like logic.” 
2. “If mathematics is easy, then logic is not difficult.” 
By translating these assumptions into statements involv- 
ing propositional variables and logical connectives, deter- 
mine whether each of the following are valid conclusions 
of these assumptions: 
a) That mathematics is not easy, if many students like 
logic. 
b) That not many students like logic, if mathematics is 
not easy. 
c) That mathematics is not easy or logic is difficult. 
d) That logic is not difficult or mathematics is not easy. 
e) That if not many students like logic, then either math- 
ematics is not easy or logic is not difficult. 
Determine whether this argument, taken from Kalish and 
Montague [KaMo64], is valid. 
If Superman were able and willing to prevent evil, 
he would do so. If Superman were unable to prevent 
evil, he would be impotent; if he were unwilling 
to prevent evil, he would be malevolent. Superman 
does not prevent evil. If Superman exists, he is nei- 
ther impotent nor malevolent. Therefore, Superman 
does not exist. 


Introduction 


In this section we introduce the notion of a proof and describe methods for constructing proofs. 
A proof is a valid argument that establishes the truth of a mathematical statement. A proof can 
use the hypotheses of the theorem, if any, axioms assumed to be true, and previously proven 
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theorems. Using these ingredients and rules of inference, the final step of the proof establishes 
the truth of the statement being proved. 

In our discussion we move from formal proofs of theorems toward more informal proofs. 
The arguments we introduced in Section 1.6 to show that statements involving propositions 
and quantified statements are true were formal proofs, where all steps were supplied, and the 
rules for each step in the argument were given. However, formal proofs of useful theorems can 
be extremely long and hard to follow. In practice, the proofs of theorems designed for human 
consumption are almost always informal proofs, where more than one rule of inference may 
be used in each step, where steps may be skipped, where the axioms being assumed and the 
rules of inference used are not explicitly stated. Informal proofs can often explain to humans 
why theorems are true, while computers are perfectly happy producing formal proofs using 
automated reasoning systems. 

The methods of proof discussed in this chapter are important not only because they are used 
to prove mathematical theorems, but also for their many applications to computer science. These 
applications include verifying that computer programs are correct, establishing that operating 
systems are secure, making inferences in artificial intelligence, showing that system specifica- 
tions are consistent, and so on. Consequently, understanding the techniques used in proofs is 
essential both in mathematics and in computer science. 


Some Terminology 


Formally, a theorem is a statement that can be shown to be true. In mathematical writing, the 
term theorem is usually reserved for a statement that is considered at least somewhat important. 
Less important theorems sometimes are called propositions. (Theorems can also be referred to 
as facts or results.) A theorem may be the universal quantification of a conditional statement 
with one or more premises and a conclusion. However, it may be some other type of logical 
statement, as the examples later in this chapter will show. We demonstrate that a theorem is true 
with a proof. A proof is a valid argument that establishes the truth of a theorem. The statements 
used in a proof can include axioms (or postulates), which are statements we assume to be true 
(for example, the axioms for the real numbers, given in Appendix 1, and the axioms of plane 
geometry), the premises, if any, of the theorem, and previously proven theorems. Axioms may 
be stated using primitive terms that do not require definition, but all other terms used in theorems 
and their proofs must be defined. Rules of inference, together with definitions of terms, are used 
to draw conclusions from other assertions, tying together the steps of a proof. In practice, the 
final step of a proof is usually just the conclusion of the theorem. However, for clarity, we will 
often recap the statement of the theorem as the final step of a proof. 

A less important theorem that is helpful in the proof of other results is called a lemma 
(plural Jemmas or lemmata). Complicated proofs are usually easier to understand when they are 
proved using a series of lemmas, where each lemma is proved individually. A corollary is a 
theorem that can be established directly from a theorem that has been proved. A conjecture is 
a statement that is being proposed to be a true statement, usually on the basis of some partial 
evidence, a heuristic argument, or the intuition of an expert. When a proof of a conjecture is 
found, the conjecture becomes a theorem. Many times conjectures are shown to be false, so they 
are not theorems. 


Understanding How Theorems Are Stated 


Before we introduce methods for proving theorems, we need to understand how many math- 
ematical theorems are stated. Many theorems assert that a property holds for all elements in 
a domain, such as the integers or the real numbers. Although the precise statement of such 
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Assessment 


theorems needs to include a universal quantifier, the standard convention in mathematics is to 
omit it. For example, the statement 


“If x > y, where x and y are positive real numbers, then x > y?.” 
really means 
“For all positive real numbers x and y, if x > y, then x? > y?.” 


Furthermore, when theorems of this type are proved, the first step of the proof usually involves 
selecting a general element of the domain. Subsequent steps show that this element has the 
property in question. Finally, universal generalization implies that the theorem holds for all 
members of the domain. 


Methods of Proving Theorems 


Proving mathematical theorems can be difficult. To construct proofs we need all available am- 
munition, including a powerful battery of different proof methods. These methods provide the 
overall approach and strategy of proofs. Understanding these methods is a key component of 
learning how to read and construct mathematical proofs. One we have chosen a proof method, 
we use axioms, definitions of terms, previously proved results, and rules of inference to com- 
plete the proof. Note that in this book we will always assume the axioms for real numbers 
found in Appendix 1. We will also assume the usual axioms whenever we prove a result about 
geometry. When you construct your own proofs, be careful not to use anything but these axioms, 
definitions, and previously proved results as facts! 

To prove a theorem of the form Vx(P(x) > Q(x)), our goal is to show that P(c) > Q(c) 
is true, where c is an arbitrary element of the domain, and then apply universal generalization. 
In this proof, we need to show that a conditional statement is true. Because of this, we now focus 
on methods that show that conditional statements are true. Recall that p — gq is true unless p is 
true but q is false. Note that to prove the statement p — q, we need only show that q is true if p 
is true. The following discussion will give the most common techniques for proving conditional 
statements. Later we will discuss methods for proving other types of statements. In this section, 
and in Section 1.8, we will develop a large arsenal of proof techniques that can be used to prove 
a wide variety of theorems. 

When you read proofs, you will often find the words “obviously” or “clearly.” These words 
indicate that steps have been omitted that the author expects the reader to be able to fill in. 
Unfortunately, this assumption is often not warranted and readers are not at all sure how to fill in 
the gaps. We will assiduously try to avoid using these words and try not to omit too many steps. 
However, if we included all steps in proofs, our proofs would often be excruciatingly long. 


Direct Proofs 


A direct proof of a conditional statement p — g is constructed when the first step is the 
assumption that p is true; subsequent steps are constructed using rules of inference, with the 
final step showing that g must also be true. A direct proof shows that a conditional statement 
P — q is true by showing that if p is true, then g must also be true, so that the combination 
p true and g false never occurs. In a direct proof, we assume that p is true and use axioms, 
definitions, and previously proven theorems, together with rules of inference, to show that g 
must also be true. You will find that direct proofs of many results are quite straightforward, with a 
fairly obvious sequence of steps leading from the hypothesis to the conclusion. However, direct 
proofs sometimes require particular insights and can be quite tricky. The first direct proofs we 
present here are quite straightforward; later in the text you will see some that are less obvious. 

We will provide examples of several different direct proofs. Before we give the first example, 
we need to define some terminology. 


DEFINITION 1 


EXAMPLE 1 
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EXAMPLE 2 


EXAMPLE 3 
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The integer n is even if there exists an integer k such that n = 2k, and n 1s odd if there exists 
an integer k such that n = 2k + 1. (Note that every integer is either even or odd, and no 
integer is both even and odd.) Two integers have the same parity when both are even or both 
are odd; they have opposite parity when one is even and the other is odd. 


Give a direct proof of the theorem “If n is an odd integer, then n? is odd.” 


Solution: Note that this theorem states Vn P((n) — Q(n)), where P(n) is “n is an odd integer” 
and Q(n) is “n~ is odd.” As we have said, we will follow the usual convention in mathematical 
proofs by showing that P(n) implies Q(n), and not explicitly using universal instantiation. To 
begin a direct proof of this theorem, we assume that the hypothesis of this conditional statement 
is true, namely, we assume that n is odd. By the definition of an odd integer, it follows that 
n = 2k +1, where k is some integer. We want to show that n” is also odd. We can square 
both sides of the equation n = 2k + 1 to obtain a new equation that expresses n*. When we do 
this, we find that n? = (2k + 1)? = 4k? + 4k + 1 = 2(2k? + 2k) + 1. By the definition of an 
odd integer, we can conclude that n? is an odd integer (it is one more than twice an integer). 
Consequently, we have proved that if n is an odd integer, then n? is an odd integer. 


Give a direct proof that if m and n are both perfect squares, then nm is also a perfect square. 
(An integer a is a perfect square if there is an integer b such that a = b.) 


Solution: To produce a direct proof of this theorem, we assume that the hypothesis of this 
conditional statement is true, namely, we assume that m and n are both perfect squares. By the 
definition of a perfect square, it follows that there are integers s and ft such that m = s” and 
n = t*. The goal of the proof is to show that mn must also be a perfect square when m and n are; 
looking ahead we see how we can show this by substituting s? for m and t? for n into mn. This 
tells us that mn = s2t?. Hence, mn = s?t? = (ss)(tt) = (st)(st) = (st)?, using commutativity 
and associativity of multiplication. By the definition of perfect square, it follows that mn is also 
a perfect square, because it is the square of st, which is an integer. We have proved that if m 
and n are both perfect squares, then mn is also a perfect square. 


Proof by Contraposition 


Direct proofs lead from the premises of a theorem to the conclusion. They begin with the 
premises, continue with a sequence of deductions, and end with the conclusion. However, we 
will see that attempts at direct proofs often reach dead ends. We need other methods of proving 
theorems of the form Vx(P(x) > Q(x)). Proofs of theorems of this type that are not direct 
proofs, that is, that do not start with the premises and end with the conclusion, are called 
indirect proofs. 

An extremely useful type of indirect proof is known as proof by contraposition. Proofs 
by contraposition make use of the fact that the conditional statement p — gq is equivalent to its 
contrapositive, -g — —p. This means that the conditional statement p — g can be proved by 
showing that its contrapositive, ~g — —p, is true. In a proof by contraposition of p > q, we 
take —q as a premise, and using axioms, definitions, and previously proven theorems, together 
with rules of inference, we show that —p must follow. We will illustrate proof by contraposition 
with two examples. These examples show that proof by contraposition can succeed when we 
cannot easily find a direct proof. 

Prove that if 1 is an integer and 3n + 2 is odd, then n is odd. 


Solution: We first attempt a direct proof. To construct a direct proof, we first assume that 3n + 2 
is an odd integer. This means that 3n + 2 = 2k + 1 for some integer k. Can we use this fact 
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to show that n is odd? We see that 3n + 1 = 2k, but there does not seem to be any direct way 
to conclude that n is odd. Because our attempt at a direct proof failed, we next try a proof by 
contraposition. 

The first step in a proof by contraposition is to assume that the conclusion of the conditional 
statement “If 3n + 2 is odd, then n is odd” is false; namely, assume that n is even. Then, by 
the definition of an even integer, n = 2k for some integer k. Substituting 2k for n, we find 
that 3n + 2 = 3(2k) +2 = 6k +2 = 2(3k + 1). This tells us that 3n + 2 is even (because it 
is a multiple of 2), and therefore not odd. This is the negation of the premise of the theorem. 
Because the negation of the conclusion of the conditional statement implies that the hypothesis 
is false, the original conditional statement is true. Our proof by contraposition succeeded; we 
have proved the theorem “Tf 37 + 2 is odd, then n is odd.” < 


Prove that if n = ab, where a and D are positive integers, then a < ,/n or b < /n. 


Solution: Because there is no obvious way of showing that a < ./n or b < ./n directly from 
the equation n = ab, where a and D are positive integers, we attempt a proof by contraposition. 

The first step in a proof by contraposition is to assume that the conclusion of the conditional 
statement “Ifn = ab, where a and b are positive integers, thena < ./norb < ./n” is false. That 
is, we assume that the statement (a < ./n) V (b < ./n) is false. Using the meaning of disjunction 
together with De Morgan’s law, we see that this implies that both a < ./n and b < ./nare false. 
This implies that a > ./n and b > ./n. We can multiply these inequalities together (using the 
fact that if 0 < s < ¢ and 0 <u <v, then su < ftv) to obtain ab > /n- ./n =n. This shows 
that ab £ n, which contradicts the statement n = ab. 

Because the negation of the conclusion of the conditional statement implies that the hypoth- 
esis is false, the original conditional statement is true. Our proof by contraposition succeeded; 
we have proved that if n = ab, where a and b are positive integers, then a < ./n or b < ./n.4 


VACUOUS AND TRIVIAL PROOFS We can quickly prove that a conditional statement 
P — q is true when we know that p is false, because p — g must be true when p is false. 
Consequently, if we can show that p is false, then we have a proof, called a vacuous proof, of 
the conditional statement p — q. Vacuous proofs are often used to establish special cases of 
theorems that state that a conditional statement is true for all positive integers [1.e., a theorem 
of the kind Vn P(n), where P(n) is a propositional function]. Proof techniques for theorems of 
this kind will be discussed in Section 5.1. 


Show that the proposition P(0) is true, where P(n) is “If n > 1, then n2 > n” and the domain 
consists of all integers. 


Solution: Note that P(0) is “If 0 > 1, then 0? > 0.’ We can show P(0) using a vacuous 
proof. Indeed, the hypothesis 0 > 1 is false. This tells us that P(0) is automatically true. < 


Remark: The fact that the conclusion of this conditional statement, 02 > 0, is false is irrelevant 
to the truth value of the conditional statement, because a conditional statement with a false 
hypothesis is guaranteed to be true. 


We can also quickly prove a conditional statement p — q if we know that the conclusion 
q is true. By showing that gq is true, it follows that p — gq must also be true. A proof of p > q 
that uses the fact that g is true is called a trivial proof. Trivial proofs are often important when 
special cases of theorems are proved (see the discussion of proof by cases in Section 1.8) and 
in mathematical induction, which is a proof technique discussed in Section 5.1. 


EXAMPLE 6 


DEFINITION 2 


EXAMPLE 7 
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Let P(n) be “If a and b are positive integers with a > b, then a” > b",” where the domain 
consists of all nonnegative integers. Show that P (0) is true. 


Solution: The proposition P (0) is “Ifa > b, then a® > b°” Because a® = b° = 1, the conclusion 
of the conditional statement “If a > b, then a° > b°” is true. Hence, this conditional statement, 
which is P(O), is true. This is an example of a trivial proof. Note that the hypothesis, which is 
the statement “a > b,” was not needed in this proof. < 


A LITTLE PROOF STRATEGY We have described two important approaches for proving 
theorems of the form Vx(P(x) — Q(x)): direct proof and proof by contraposition. We have 
also given examples that show how each is used. However, when you are presented with a 
theorem of the form Vx(P(x) — Q(x)), which method should you use to attempt to prove it? 
We will provide a few rules of thumb here; in Section 1.8 we will discuss proof strategy at greater 
length. When you want to prove a statement of the form Vx(P(x) > Q(x)), first evaluate 
whether a direct proof looks promising. Begin by expanding the definitions in the hypotheses. 
Start to reason using these hypotheses, together with axioms and available theorems. If a direct 
proof does not seem to go anywhere, try the same thing with a proof by contraposition. Recall 
that in a proof by contraposition you assume that the conclusion of the conditional statement is 
false and use a direct proof to show this implies that the hypothesis must be false. We illustrate 
this strategy in Examples 7 and 8. Before we present our next example, we need a definition. 


The real number r is rational if there exist integers p and g with g € 0 such that r = p/q. 
A real number that is not rational is called irrational. 


Prove that the sum of two rational numbers is rational. (Note that if we include the implicit 
quantifiers here, the theorem we want to prove is “For every real number r and every real 
number s, if r and s are rational numbers, then r + s is rational.) 


Solution: We first attempt a direct proof. To begin, suppose that r and s are rational numbers. From 
the definition of a rational number, it follows that there are integers p and q, with g € 0, such 
that r = p/q, and integers t and u, with u 4 0, such that s = t/u. Can we use this information 
to show that r + s is rational? The obvious next step is to add r = p/q and s = t/u, to obtain 


t t 
pang ee oy cael bag 
q u qu 


Because g # 0 and u # 0, it follows that qu 4 0. Consequently, we have expressed r + s as 
the ratio of two integers, pu + qt and qu, where qu 4 0. This means that r + s is rational. We 
have proved that the sum of two rational numbers is rational; our attempt to find a direct proof 
succeeded. 


Prove that if n is an integer and n? is odd, then n is odd. 


Solution: We first attempt a direct proof. Suppose that n is an integer and n? is odd. Then, there 
exists an integer k such that n? = 2k + 1. Can we use this information to show that n is odd? 
There seems to be no obvious approach to show that n is odd because solving for n produces 
the equation n = +./2k + 1, which is not terribly useful. 

Because this attempt to use a direct proof did not bear fruit, we next attempt a proof by 
contraposition. We take as our hypothesis the statement that n is not odd. Because every integer 
is odd or even, this means that n is even. This implies that there exists an integer k such that 
n = 2k. To prove the theorem, we need to show that this hypothesis implies the conclusion 
that n? is not odd, that is, that n? is even. Can we use the equation n = 2k to achieve this? By 
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squaring both sides of this equation, we obtain n2 = 4k? = 2(2k*), which implies that n? is 
also even because n? = 2t, where t = 2k*. We have proved that if n is an integer and n? is odd, 
then n is odd. Our attempt to find a proof by contraposition succeeded. < 


Proofs by Contradiction 


Suppose we want to prove that a statement p is true. Furthermore, suppose that we can find 
a contradiction gq such that ~p — q is true. Because g is false, but ~p — gq is true, we can 
conclude that —p is false, which means that p is true. How can we find a contradiction g that 
might help us prove that p is true in this way? 

Because the statement r A —r is a contradiction whenever r is a proposition, we can prove 
that p is true if we can show that —p — (r A —r) is true for some proposition r. Proofs of this 
type are called proofs by contradiction. Because a proof by contradiction does not prove a result 
directly, it is another type of indirect proof. We provide three examples of proof by contradiction. 
The first is an example of an application of the pigeonhole principle, a combinatorial technique 
that we will cover in depth in Section 6.2. 


Show that at least four of any 22 days must fall on the same day of the week. 


Solution: Let p be the proposition “At least four of 22 chosen days fall on the same day of the 
week.” Suppose that —p is true. This means that at most three of the 22 days fall on the same 
day of the week. Because there are seven days of the week, this implies that at most 21 days 
could have been chosen, as for each of the days of the week, at most three of the chosen days 
could fall on that day. This contradicts the premise that we have 22 days under consideration. 
That is, if r is the statement that 22 days are chosen, then we have shown that —p —> (r A =r). 
Consequently, we know that p is true. We have proved that at least four of 22 chosen days fall 
on the same day of the week. 4 


Prove that /2 is irrational by giving a proof by contradiction. 


Solution: Let p be the proposition “4/2 is irrational.” To start a proof by contradiction, we suppose 
that —p is true. Note that sp is the statement “It is not the case that ./2 is irrational,” which 
says that /2 is rational. We will show that assuming that —p is true leads to a contradiction. 

If /2 is rational, there exist integers a and b with /2 = a/b, where b £ 0 and a and b 
have no common factors (so that the fraction a/b is in lowest terms.) (Here, we are using the 
fact that every rational number can be written in lowest terms.) Because 2 = a/b, when both 
sides of this equation are squared, it follows that 


a 
2 = RB 
Hence, 
2b? =a’. 


By the definition of an even integer it follows that a? is even. We next use the fact that if a? is 
even, a must also be even, which follows by Exercise 16. Furthermore, because a is even, by 
the definition of an even integer, a = 2c for some integer c. Thus, 


ee om 
Dividing both sides of this equation by 2 gives 
Pate, 


By the definition of even, this means that b? is even. Again using the fact that if the square of an 
integer is even, then the integer itself must be even, we conclude that b must be even as well. 
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We have now shown that the assumption of =p leads to the equation /2 = a/b, where a 
and b have no common factors, but both a and b are even, that is, 2 divides both a and b. Note 
that the statement that /2 = a /b, where a and b have no common factors, means, in particular, 
that 2 does not divide both a and b. Because our assumption of —p leads to the contradiction 
that 2 divides both a and b and 2 does not divide both a and b, —p must be false. That is, the 
statement p, “«/2 is irrational,” is true. We have proved that ./2 is irrational. < 


Proof by contradiction can be used to prove conditional statements. In such proofs, we first 
assume the negation of the conclusion. We then use the premises of the theorem and the negation 
of the conclusion to arrive at a contradiction. (The reason that such proofs are valid rests on the 
logical equivalence of p — qg and (p A -=q) — F. To see that these statements are equivalent, 
simply note that each is false in exactly one case, namely when p is true and g is false.) 

Note that we can rewrite a proof by contraposition of a conditional statement as a proof 
by contradiction. In a proof of p — q by contraposition, we assume that —g is true. We then 
show that =p must also be true. To rewrite a proof by contraposition of p — q as a proof by 
contradiction, we suppose that both p and —¢ are true. Then, we use the steps from the proof 
of —q — —p to show that —p is true. This leads to the contradiction p A =p, completing the 
proof. Example 11 illustrates how a proof by contraposition of a conditional statement can be 
rewritten as a proof by contradiction. 


EXAMPLE 11 _ Give a proof by contradiction of the theorem “If 3n + 2 is odd, then n is odd.” 


Solution: Let p be “3n + 2 is odd” and g be “n is odd.” To construct a proof by contradiction, 
assume that both p and —g are true. That is, assume that 37 + 2 is odd and that n is not odd. 
Because n is not odd, we know that it is even. Because n is even, there is an integer k such 
that n = 2k. This implies that 3n + 2 = 3(2k) + 2 = 6k +2 = 2(3k + 1). Because 3n + 2 is 
2t, where t = 3k + 1, 3n + 2 1s even. Note that the statement “3n + 2 is even” is equivalent to 
the statement —p, because an integer is even if and only if it is not odd. Because both p and 
—p are true, we have a contradiction. This completes the proof by contradiction, proving that if 
3n + 2 is odd, then n is odd. 


Note that we can also prove by contradiction that p — gq is true by assuming that p and 
—q are true, and showing that g must be also be true. This implies that —g and qg are both 
true, a contradiction. This observation tells us that we can turn a direct proof into a proof by 
contradiction. 


PROOFS OF EQUIVALENCE | To prove a theorem that is a biconditional statement, that is, 
a statement of the form p < g, we show that p — g and q — p are both true. The validity of 
this approach is based on the tautology 


(Pog (P>rqQAG- P). 


EXAMPLE 12 Prove the theorem “If n is an integer, then n is odd if and only if n? is odd.” 

Solution: This theorem has the form “p if and only if g,” where p is “n is odd” and q is “n? 
is odd.” (As usual, we do not explicitly deal with the universal quantification.) To prove this 
theorem, we need to show that p > g and gq — p are true. 

Extra ~ We have already shown (in Example 1) that p — q is true and (in Example 8) that g > p 
Examples @ixd is true. 
Because we have shown that both p — qg and g — p are true, we have shown that the 
theorem is true. 
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Sometimes a theorem states that several propositions are equivalent. Such a theorem states 
that propositions pj, p2, p3,--., Pn are equivalent. This can be written as 


Pl <> P2 <> °+* <> Pn, 


which states that all n propositions have the same truth values, and consequently, that for all i 
and j with} <i <nand1 < j <n, p; and p; are equivalent. One way to prove these mutually 
equivalent is to use the tautology 


Di <> pr +++ > Pn @ (Pi > p2) A (p2 > p3) A+++ A (Pn > Pi)- 


This shows that if the n conditional statements pj > p2, p2 > P3,---, Pn > p1 canbe shown 
to be true, then the propositions p), p2,..., Pn are all equivalent. 

This is much more efficient than proving that p; > pj; for alli 4 j with 1 <i <n and 
1 < j <n. (Note that there are n? —n such conditional statements.) 

When we prove that a group of statements are equivalent, we can establish any chain of 
conditional statements we choose as long as it is possible to work through the chain to go from 
any one of these statements to any other statement. For example, we can show that p1, p2, and 
p3 are equivalent by showing that py > p3, p3 > p2, and p2 > pi. 


EXAMPLE 13 _ Show that these statements about the integer n are equivalent: 


Pi: nis even. 
p2: n-— 1is odd. 


p3: n? is even. 


Solution: We will show that these three statements are equivalent by showing that the conditional 
statements pj > p2, p2 > p3, and p3 > py, are true. 

We use a direct proof to show that p; — p2. Suppose that n is even. Then n = 2k for some 
integer k. Consequently, n — 1 = 2k — 1 = 2(k — 1) + 1. This means that n — 1 is odd because 
it is of the form 2m + 1, where m is the integer k — 1. 

We also use a direct proof to show that p2 — p3. Now suppose n — | is odd. Then n — 
1 = 2k + 1 for some integer k. Hence, n = 2k + 2 so that n? = (2k +2)? = 4k? 4+ 8k4+4= 
2(2k? + 4k + 2). This means that n? is twice the integer 2k? + 4k + 2, and hence is even. 

To prove p3 > p1, we use a proof by contraposition. That is, we prove that if n is not even, 
then n? is not even. This is the same as proving that if n is odd, then n? is odd, which we have 
already done in Example |. This completes the proof. < 


COUNTEREXAMPLES In Section 1.4 we stated that to show that a statement of the form 
Vx P(x) is false, we need only find a counterexample, that is, an example x for which P(x) 
is false. When presented with a statement of the form Vx P(x), which we believe to be false or 
which has resisted all proof attempts, we look for a counterexample. We illustrate the use of 
counterexamples in Example 14. 


EXAMPLE 14 — Show that the statement “Every positive integer is the sum of the squares of two integers” is 
false. 


Solution: To show that this statement is false, we look for a counterexample, which is a particular 

Extra a integer that is not the sum of the squares of two integers. It does not take long to find a counterex- 
Examples Qi ample, because 3 cannot be written as the sum of the squares of two integers. To show this is the 
case, note that the only perfect squares not exceeding 3 are 0* = 0 and 1? = 1. Furthermore, 

there is no way to get 3 as the sum of two terms each of which is 0 or 1. Consequently, we have 

shown that “Every positive integer is the sum of the squares of two integers” is false. < 


Links 


EXAMPLE 15 


EXAMPLE 16 


EXAMPLE 17 
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Mistakes in Proofs 


There are many common errors made in constructing mathematical proofs. We will briefly 
describe some of these here. Among the most common errors are mistakes in arithmetic and basic 
algebra. Even professional mathematicians make such errors, especially when working with 
complicated formulae. Whenever you use such computations you should check them as carefully 
as possible. (You should also review any troublesome aspects of basic algebra, especially before 
you study Section 5.1.) 

Each step of a mathematical proof needs to be correct and the conclusion needs to follow 
logically from the steps that precede it. Many mistakes result from the introduction of steps that 
do not logically follow from those that precede it. This is illustrated in Examples 15-17. 


What is wrong with this famous supposed “proof” that 1 = 2? 


“Proof:” We use these steps, where a and b are two equal positive integers. 


Step Reason 

loa= Given 

2. a* =ab Multiply both sides of (1) by a 

3. a —b? =ab—-b* Subtract b? from both sides of (2) 

4. (a—b)(a+b) = b(a —b) Factor both sides of (3) 

5.at+tb=b Divide both sides of (4) by a — b 

6. 2b=b Replace a by b in (5) because a = b 
and simplify 

7 ae a | Divide both sides of (6) by b 


Solution: Every step is valid except for one, step 5 where we divided both sides by a — b. The 
error is that a — b equals zero; division of both sides of an equation by the same quantity is 
valid as long as this quantity is not zero. 


What is wrong with this “proof?” 


“Theorem:” If n? is positive, then n is positive. 


“Proof:” Suppose that n? is positive. Because the conditional statement “If n is positive, then 


n? is positive” is true, we can conclude that n is positive. 


Solution: Let P(n) be “n is positive” and Q(n) be “n“ is positive.” Then our hypothesis is O(n). 
The statement “If is positive, then n? is positive” is the statement Vn(P(n) > Q(n)). From 
the hypothesis Q(n) and the statement Vn(P(n) — Q(n)) we cannot conclude P(n), because 
we are not using a valid rule of inference. Instead, this is an example of the fallacy of affirming 
the conclusion. A counterexample is supplied by n = —1 for which n? = 1 is positive, but n is 
negative. < 


What is wrong with this “proof?” 


“Theorem:” If n is not positive, then n7 is not positive. (This is the contrapositive of the 
“theorem” in Example 16.) 
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“Proof:” Suppose that n is not positive. Because the conditional statement “Tf n is positive, then 
n? is positive” is true, we can conclude that n? is not positive. 


Solution: Let P(n) and Q(n) be as in the solution of Example 16. Then our hypothesis is =P (n) 
and the statement “If 7 is positive, then n? is positive” is the statement Vn(P(n) > Q(n)). 
From the hypothesis —P(n) and the statement Vn(P(n) — Q(n)) we cannot conclude —Q(n), 
because we are not using a valid rule of inference. Instead, this is an example of the fallacy of 
denying the hypothesis. A counterexample is supplied by n = —1, as in Example 16. < 


Finally, we briefly discuss a particularly nasty type of error. Many incorrect arguments are 
based on a fallacy called begging the question. This fallacy occurs when one or more steps of 
a proof are based on the truth of the statement being proved. In other words, this fallacy arises 
when a statement is proved using itself, or a statement equivalent to it. That is why this fallacy 
is also called circular reasoning. 


Is the following argument correct? It supposedly shows that n is an even integer whenever n? is 


an even integer. 


Suppose that n? is even. Then n* = 2k for some integer k. Let n = 2/ for some integer J. 
This shows that n is even. 


Solution: This argument is incorrect. The statement “let n = 2/ for some integer 7” occurs in 
the proof. No argument has been given to show that n can be written as 2/ for some integer /. 
This is circular reasoning because this statement is equivalent to the statement being proved, 
namely, “n is even.” Of course, the result itself is correct; only the method of proof is wrong.4 


Making mistakes in proofs is part of the learning process. When you make a mistake that 
someone else finds, you should carefully analyze where you went wrong and make sure that 
you do not make the same mistake again. Even professional mathematicians make mistakes in 
proofs. More than a few incorrect proofs of important results have fooled people for many years 
before subtle errors in them were found. 


Just a Beginning 


We have now developed a basic arsenal of proof methods. In the next section we will introduce 
other important proof methods. We will also introduce several important proof techniques in 
Chapter 5, including mathematical induction, which can be used to prove results that hold for 
all positive integers. In Chapter 6 we will introduce the notion of combinatorial proofs. 

In this section we introduced several methods for proving theorems of the form Vx (P(x) > 
Q(x)), including direct proofs and proofs by contraposition. There are many theorems of this 
type whose proofs are easy to construct by directly working through the hypotheses and def- 
initions of the terms of the theorem. However, it is often difficult to prove a theorem without 
resorting to a clever use of a proof by contraposition or a proof by contradiction, or some 
other proof technique. In Section 1.8 we will address proof strategy. We will describe various 
approaches that can be used to find proofs when straightforward approaches do not work. Con- 
structing proofs is an art that can be learned only through experience, including writing proofs, 
having your proofs critiqued, and reading and analyzing other proofs. 


c= 


Exercises 


18. 


19. 


20. 


21. 


22. 


. Use a direct proof to show that the sum of two odd integers 


is even. 


. Use a direct proof to show that the sum of two even inte- 


gers is even. 


. Show that the square of an even number is an even number 


using a direct proof. 


. Show that the additive inverse, or negative, of an even 


number is an even number using a direct proof. 


. Prove that if m +n and n+ p are even integers, where 


m,n, and p are integers, then m + p is even. What kind 
of proof did you use? 


. Use a direct proof to show that the product of two odd 


numbers is odd. 


. Use a direct proof to show that every odd integer is the 


difference of two squares. 


. Prove that if n is a perfect square, then n + 2 is not a 


perfect square. 


. Use a proof by contradiction to prove that the sum of an 


irrational number and a rational number is irrational. 


. Use a direct proof to show that the product of two rational 


numbers is rational. 


. Prove or disprove that the product of two irrational num- 


bers is irrational. 


. Prove or disprove that the product of a nonzero rational 


number and an irrational number is irrational. 


. Prove that if x is irrational, then 1 /x is irrational. 
. Prove that if x is rational and x ¥ 0, then 1/x is rational. 
. Use a proof by contraposition to show that if x + y > 2, 


where x and y are real numbers, then x > 1 or y > 1. 


. Prove that if m and n are integers and mn is even, then m 


is even or 7 is even. 


. Show that if n is an integer and ne +5 is odd, then n is 


even using 


a) a proof by contraposition. 
b) a proof by contradiction. 


Prove that if n is an integer and 3n + 2 is even, then n is 
even using 

a) a proof by contraposition. 

b) a proof by contradiction. 

Prove the proposition P(0), where P(n) is the proposi- 
tion “If n is a positive integer greater than 1, thenn” > n.” 
What kind of proof did you use? 

Prove the proposition P(1), where P(7) is the proposi- 
tion “If 1 is a positive integer, then n2 > n.” What kind 
of proof did you use? 

Let P(n) be the proposition “If a and b are positive real 
numbers, then (a + b)” > a" + b”.” Prove that P(1) is 
true. What kind of proof did you use? 

Show that if you pick three socks from a drawer contain- 
ing just blue socks and black socks, you must get either 
a pair of blue socks or a pair of black socks. 


23. 


24. 


25. 


26. 


27. 


28. 
29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


37. 
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Show that at least ten of any 64 days chosen must fall on 
the same day of the week. 


Show that at least three of any 25 days chosen must fall 
in the same month of the year. 


Use a proof by contradiction to show that there is no ratio- 
nal number r for which r? + r + 1 = 0. [Hint: Assume 
that r = a/b is aroot, where a and b are integers and a/b 
is in lowest terms. Obtain an equation involving integers 
by multiplying by b?. Then look at whether a and b are 
each odd or even. ] 


Prove that if n is a positive integer, then n is even if and 
only if 7n + 4 is even. 
Prove that if n is a positive integer, then is odd if and 
only if 5n + 6 is odd. 


Prove that m? = n? if and only if m =n orm = —n. 
Prove or disprove that if m and n are integers such that 
mn = |, then either m = 1 and n = 1, or else m = —1 
andn = —1. 


Show that these three statements are equivalent, where a 
and b are real numbers: (i) a is less than b, (ii) the average 
of a and b is greater than a, and (iii) the average of a and 
b is less than b. 


Show that these statements about the integer x are equiv- 
alent: (7) 3x + 2 is even, (ii) x + 5 is odd, (iii) x? is even. 
Show that these statements about the real number x are 
equivalent: (7) x is rational, (ii) x /2 is rational, (iii) 3x — 1 
is rational. 

Show that these statements about the real number x are 
equivalent: (i) x is irrational, (ii) 3x +2 is irrational, 
(iii) x /2 is irrational. 

Is this reasoning for finding the solutions of the equa- 
tion /2x2 — 1 = x correct? (1) /2x?2 — 1 = x is given; 
(2) 2x? — 1 = x?, obtained by squaring both sides of (1); 
(3) x? — 1 =0, obtained by subtracting x? from both 
sides of (2); (4) (x — 1)(x + 1) = O, obtained by factor- 
ing the left-hand side of x2 1;(5)x=lorx=-l, 
which follows because ab = 0 implies that a = 0 or 
b=0. 

Are these steps for finding the solutions of /x +3 = 
3 — x correct? J) /x + 3 = 3 — x is given; (2)x +3 = 
x? — 6x +9, obtained by squaring both sides of (1); (3) 
0 =x*—7x +6, obtained by subtracting x +3 from 
both sides of (2); (4) 0 = (« — 1)(x — 6), obtained by 
factoring the right-hand side of (3); (5) x = 1 or x = 6, 
which follows from (4) because ab = 0 implies that 
a=0Oorb=0. 

Show that the propositions pi, p2, p3, and p4 can be 
shown to be equivalent by showing that pj = pa, p2 = 
p3, and pi, <> p3. 

Show that the propositions pj, p2, p3, pa, and ps can 
be shown to be equivalent by proving that the conditional 


statements pj > p4, P3 > P1,P4 > P2, P2 > Ps, and 
Ps — p3 are true. 
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Find a counterexample to the statement that every posi- 
tive integer can be written as the sum of the squares of 
three integers. 

Prove that at least one of the real numbers a1, a2, ..., dn 
is greater than or equal to the average of these numbers. 
What kind of proof did you use? 

Use Exercise 39 to show that if the first 10 positive inte- 
gers are placed around a circle, in any order, there exist 


41. 


42. 


three integers in consecutive locations around the circle 
that have a sum greater than or equal to 17. 

Prove that if is an integer, these four statements are 
equivalent: (i) n is even, (ii) n + 1 is odd, (iii) 3n + 1 is 
odd, (iv) 3n is even. 

Prove that these four statements about the integer n are 
equivalent: (7) n2 is odd, (ii) 1 — n is even, (iii) n3 is odd, 
(iv) n? + 1 is even. 


Introduction 


Assessment 


In Section 1.7 we introduced many methods of proof and illustrated how each method can be 
used. In this section we continue this effort. We will introduce several other commonly used proof 


methods, including the method of proving a theorem by considering different cases separately. 
We will also discuss proofs where we prove the existence of objects with desired properties. 

In Section 1.7 we briefly discussed the strategy behind constructing proofs. This strategy 
includes selecting a proof method and then successfully constructing an argument step by step, 
based on this method. In this section, after we have developed a versatile arsenal of proof 
methods, we will study some aspects of the art and science of proofs. We will provide advice 
on how to find a proof of a theorem. We will describe some tricks of the trade, including how 
proofs can be found by working backward and by adapting existing proofs. 

When mathematicians work, they formulate conjectures and attempt to prove or disprove 
them. We will briefly describe this process here by proving results about tiling checkerboards 
with dominoes and other types of pieces. Looking at tilings of this kind, we will be able to 
quickly formulate conjectures and prove theorems without first developing a theory. 

We will conclude the section by discussing the role of open questions. In particular, we 
will discuss some interesting problems either that have been solved after remaining open for 


hundreds of years or that still remain open. 


Exhaustive Proof and Proof by Cases 


Sometimes we cannot prove a theorem using a single argument that holds for all possible cases. 
We now introduce a method that can be used to prove a theorem, by considering different cases 
separately. This method is based on a rule of inference that we will now introduce. To prove a 


conditional statement of the form 


(P1V p2V-+*V Pn) > 4 


the tautology 


(piv poV---V pr) > ae lpi > MA (27> AAA (Pn > Q)) 


can be used as a rule of inference. This shows that the original conditional statement with 


a hypothesis made up of a disjunction of the propositions p1, p2,... 


, Pn can be proved by 


proving each of the n conditional statements p; > g, i = 1,2,...,n, individually. Such an 
argument is called a proof by cases. Sometimes to prove that a conditional statement p > q is 
true, it is convenient to use a disjunction p; V p2 V --- V Pp instead of p as the hypothesis of 
the conditional statement, where p and p; V p2 V--- V pp are equivalent. 
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EXHAUSTIVE PROOF Some theorems can be proved by examining arelatively small number 
of examples. Such proofs are called exhaustive proofs, or proofs by exhaustion because these 
proofs proceed by exhausting all possibilities. An exhaustive proof is a special type of proof by 
cases where each case involves checking a single example. We now provide some illustrations 
of exhaustive proofs. 


EXAMPLE 1 Prove that (n + 1)? > 3” ifn is a positive integer with n < 4. 
Solution: We use a proof by exhaustion. We only need verify the inequality (n + 1)? > 3” 


when n = 1, 2,3, and 4. For n = 1, we have (n + 1)° = 23=8and3"=3! = 3; for n = 2, 
we have (n + 1)? = 3° = 27 and 3” = 3? = 9; for n = 3, we have (n+ 1)? = 4 = 64 and 


Extra & 3” = 33 = 27; and for n = 4, we have (n + 1)? = 5° = 125 and 3” = 34 = 81. In each of 
Examples Qk these four cases, we see that (n + 1)? > 3”. We have used the method of exhaustion to prove 
that (n + 1)? > 3” ifn is a positive integer with n < 4. < 


EXAMPLE 2 | Prove that the only consecutive positive integers not exceeding 100 that are perfect powers are 
8 and 9. (An integer is a perfect power if it equals n“, where a is an integer greater than 1.) 


Solution: We use a proof by exhaustion. In particular, we can prove this fact by examining 
positive integers n not exceeding 100, first checking whether n is a perfect power, and if it is, 
checking whether n + | is also a perfect power. A quicker way to do this is simply to look at all 
perfect powers not exceeding 100 and checking whether the next largest integer is also a perfect 
power. The squares of positive integers not exceeding 100 are 1, 4, 9, 16, 25, 36, 49, 64, 81, and 
100. The cubes of positive integers not exceeding 100 are 1, 8, 27, and 64. The fourth powers 
of positive integers not exceeding 100 are 1, 16, and 81. The fifth powers of positive integers 
not exceeding 100 are | and 32. The sixth powers of positive integers not exceeding 100 are | 
and 64. There are no powers of positive integers higher than the sixth power not exceeding 100, 
other than 1. Looking at this list of perfect powers not exceeding 100, we see that n = 8 is the 
only perfect power n for which n + 1 is also a perfect power. That is, 2? = 8 and 3* = 9 are the 
only two consecutive perfect powers not exceeding 100. 


Proofs by exhaustion can 


ieauipensieaed People can carry out exhaustive proofs when it is necessary to check only a relatively small 


computers when the number of instances of a statement. Computers do not complain when they are asked to check 
number of cases a much larger number of instances of a statement, but they still have limitations. Note that not 
challenges the available even a computer can check all instances when it is impossible to list all instances to check. 


processing power! 


PROOF BY CASES _ A proof by cases must cover all possible cases that arise in a theorem. 
We illustrate proof by cases with a couple of examples. In each example, you should check that 
all possible cases are covered. 


EXAMPLE 3 Prove that if n is an integer, then n? > n. 


Solution: We can prove that n* > n for every integer by considering three cases, when n = 0, 
whenn > 1, and whenn < —1. We split the proof into three cases because it is straightforward 
to prove the result by considering zero, positive integers, and negative integers separately. 


Case (i): When n = O, because 0? = 0, we see that 0? > 0. It follows that n2 > n is true in 


this case. 
Extra a Case (ii): When n > 1, when we multiply both sides of the inequality n > 1 by the positive 
Examples @ixd integer n, we obtain n -n > n- 1. This implies that n2>nforn> 1. 


Case (iii): In this case n < —1. However, n2 > 0. It follows that n2 >n. 


Because the inequality n* > n holds in all three cases, we can conclude that if n is an integer, 
then n? >n. 
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EXAMPLE 4 


EXAMPLE 5 


Use a proof by cases to show that |xy| = |x||y|, where x and y are real numbers. (Recall that 
|a|, the absolute value of a, equals a when a > 0 and equals —a when a < 0.) 


Solution: In our proof of this theorem, we remove absolute values using the fact that |a| = a 
when a > 0 and |a| = —a when a < 0. Because both |x| and | y| occur in our formula, we will 
need four cases: (i) x and y both nonnegative, (ii) x nonnegative and y is negative, (iii) x negative 
and y nonnegative, and (iv) x negative and y negative. We denote by pj, p2, p3, and p4, the 
proposition stating the assumption for each of these four cases, respectively. 

(Note that we can remove the absolute value signs by making the appropriate choice of 
signs within each case.) 


Case (i): We see that py — q because xy > 0 when x > Oand y > 0, so that |xy| = xy = 
Ix||yI. 

Case (ii): To see that p2 — q, note that if x > 0 and y < 0, then xy < 0, so that |xy| = 
—xy = x(-y) = |x||y|. (Here, because y < 0, we have |y| = —y.) 

Case (iii): To see that p3 — gq, we follow the same reasoning as the previous case with the 
roles of x and y reversed. 

Case (iv): To see that p4 — q, note that when x < 0 and y < 0, it follows that xy > 0. 
Hence, |xy| = xy = (—x)(—y) = [allyl- 


Because |xy| = |x||y| holds in each of the four cases and these cases exhaust all possibilities, 
we can conclude that |xy| = |x||y|, whenever x and y are real numbers. < 


LEVERAGING PROOF BY CASES The examples we have presented illustrating proof by 
cases provide some insight into when to use this method of proof. In particular, when it is not 
possible to consider all cases of a proof at the same time, a proof by cases should be considered. 
When should you use such a proof? Generally, look for a proof by cases when there is no 
obvious way to begin a proof, but when extra information in each case helps move the proof 
forward. Example 5 illustrates how the method of proof by cases can be used effectively. 


Formulate a conjecture about the final decimal digit of the square of an integer and prove your 
result. 


Solution: The smallest perfect squares are 1, 4,9, 16, 25, 36, 49, 64, 81, 100, 121, 144, 169, 
196, 225, and so on. We notice that the digits that occur as the final digit of a square are 
0, 1, 4,5, 6, and 9, with 2, 3, 7, and 8 never appearing as the final digit of a square. We conjecture 
this theorem: The final decimal digit of a perfect square is 0, 1, 4, 5, 6 or 9. How can we prove 
this theorem? 

We first note that we can express an integer n as 10a +b, where a and D are pos- 
itive integers and Db is 0,1,2,3,4,5,6,7,8, or 9. Here a is the integer obtained by 
subtracting the final decimal digit of n from n and dividing by 10. Next, note that 
(10a + b)* = 100a? + 20ab + b* = 10(10a” + 2b) + b?, so that the final decimal digit of n? 
is the same as the final decimal digit of b. Furthermore, note that the final decimal digit of b? 
is the same as the final decimal digit of (10 — b)? = 100 — 20b + b*. Consequently, we can 
reduce our proof to the consideration of six cases. 


Case (i): The final digit of n is 1 or 9. Then the final decimal digit of n? is the final decimal 
digit of 1* = 1 or 9* = 81, namely 1. 

Case (ii): The final digit of n is 2 or 8. Then the final decimal digit of n? is the final decimal 
digit of 27 = 4 or 8* = 64, namely 4. 

Case (iii): The final digit of n is 3 or 7. Then the final decimal digit of n? is the final decimal 
digit of 3* = 9 or 7* = 49, namely 9. 

Case (iv): The final digit of n is 4 or 6. Then the final decimal digit of n? is the final decimal 
digit of 47 = 16 or 6* = 36, namely 6. 

Case (v): The final decimal digit of n is 5. Then the final decimal digit of n? is the final 
decimal digit of 57 = 25, namely 5. 


EXAMPLE 6 


? 


In a proof by cases be 
sure not to omit any cases 
and check that you have 
proved all cases correctly! 


EXAMPLE 7 
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Case (vi): The final decimal digit of n is 0. Then the final decimal digit of n? is the final 
decimal digit of 0? = 0, namely 0. 


Because we have considered all six cases, we can conclude that the final decimal digit of n?, 
where n is an integer is either 0, 1, 2, 4, 5, 6, or 9. < 


Sometimes we can eliminate all but a few examples in a proof by cases, as Example 6 
illustrates. 


Show that there are no solutions in integers x and y of x? + 3y* = 8. 


Solution: We can quickly reduce a proof to checking just a few simple cases because x* > 8 
when |x| > 3 and 3y" > 8 when |y| > 2. This leaves the cases when x equals —2, —1, 0, 1, 
or 2 and y equals —1, 0, or 1. We can finish using an exhaustive proof. To dispense with the 
remaining cases, we note that possible values for x* are 0, 1, and 4, and possible values for 3-y~ 
are 0 and 3, and the largest sum of possible values for x* and 3y? is 7. Consequently, it is 
impossible for x? + 3y* = 8 to hold when x and y are integers. 


WITHOUT LOSS OF GENERALITY In the proof in Example 4, we dismissed case (iii), 
where x < 0 and y > 0, because it is the same as case (ii), where x > 0 and y < 0, with the 
roles of x and y reversed. To shorten the proof, we could have proved cases (ii) and (iii) together 
by assuming, without loss of generality, that x > 0 and y < 0. Implicit in this statement is that 
we can complete the case with x < 0 and y > O using the same argument as we used for the 
case with x > 0 and y < 0, but with the obvious changes. 

In general, when the phrase “without loss of generality” is used in a proof (often abbreviated 
as WLOG), we assert that by proving one case of a theorem, no additional argument is required 
to prove other specified cases. That is, other cases follow by making straightforward changes 
to the argument, or by filling in some straightforward initial step. Proofs by cases can often be 
made much more efficient when the notion of without loss of generality is employed. Of course, 
incorrect use of this principle can lead to unfortunate errors. Sometimes assumptions are made 
that lead to a loss in generality. Such assumptions can be made that do not take into account 
that one case may be substantially different from others. This can lead to an incomplete, and 
possibly unsalvageable, proof. In fact, many incorrect proofs of famous theorems turned out 
to rely on arguments that used the idea of “without loss of generality” to establish cases that 
could not be quickly proved from simpler cases. 

We now illustrate a proof where without loss of generality is used effectively together with 
other proof techniques. 


Show that if x and y are integers and both xy and x + y are even, then both x and y are even. 


Solution: We will use proof by contraposition, the notion of without loss of generality, and proof 
by cases. First, suppose that x and y are not both even. That is, assume that x is odd or that y is 
odd (or both). Without loss of generality, we assume that x is odd, so that x = 2m + | for some 
integer k. 

To complete the proof, we need to show that xy is odd or x + y is odd. Consider 
two cases: (i) y even, and (i1) y odd. In (i), y = 2” for some integer n, so that x + y= 
(2m + 1) +2n = 2(m+n) + 1 is odd. In (ii), y = 2n + 1 for some integer n, so that xy = 
(2m + 1)(2n + 1) = 4mn + 2m + 2n + 1 = 2(2mn +m +n) + | is odd. This completes the 
proof by contraposition. (Note that our use of without loss of generality within the proof is 
justified because the proof when y is odd can be obtained by simply interchanging the roles of 
x and y in the proof we have given.) 


COMMON ERRORS WITH EXHAUSTIVE PROOF AND PROOF BY CASES Acommon 
error of reasoning is to draw incorrect conclusions from examples. No matter how many separate 
examples are considered, a theorem is not proved by considering examples unless every possible 
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EXAMPLE 8 


EXAMPLE 9 


EXAMPLE 10 


Extra 
Examples 


~@ 


case is covered. The problem of proving a theorem is analogous to showing that a computer 
program always produces the output desired. No matter how many input values are tested, unless 
all input values are tested, we cannot conclude that the program always produces the correct 
output. 


Is it true that every positive integer is the sum of 18 fourth powers of integers? 


Solution: To determine whether a positive integer n can be written as the sum of 18 fourth powers 
of integers, we might begin by examining whether n is the sum of 18 fourth powers of integers 
for the smallest positive integers. Because the fourth powers of integers are 0, 1, 16, 81,..., 
if we can select 18 terms from these numbers that add up to n, then n is the sum of 18 fourth 
powers. We can show that all positive integers up to 78 can be written as the sum of 18 fourth 
powers. (The details are left to the reader.) However, if we decided this was enough checking, 
we would come to the wrong conclusion. It is not true that every positive integer is the sum of 
18 fourth powers because 79 is not the sum of 18 fourth powers (as the reader can verify). <4 


Another common error involves making unwarranted assumptions that lead to incorrect 
proofs by cases where not all cases are considered. This is illustrated in Example 9. 


What is wrong with this “proof?” 


“Theorem:” If x is a real number, then x2 


is a positive real number. 

“Proof:” Let p, be “x is positive,” let p2 be “x is negative,” and let g be “x? is positive.” To 
show that p, — q is true, note that when x is positive, x? is positive because it is the product 
of two positive numbers, x and x. To show that p2 — q, note that when x is negative, x? is 
positive because it is the product of two negative numbers, x and x. This completes the proof. 


Solution: The problem with this “proof” is that we missed the case of x = 0. When x = 0, 
x? = 0 is not positive, so the supposed theorem is false. If p is “x is a real number,” then 
we can prove results where p is the hypothesis with three cases, p}, p2, and p3, where 
pi is “x is positive,” p2 is “x is negative,” and p3 is “x = 0” because of the equivalence 
p< PIV p2V ps. < 


Existence Proofs 


Many theorems are assertions that objects of a particular type exist. A theorem of this type is a 
proposition of the form 4x P(x), where P is a predicate. A proof of a proposition of the form 
dx P(x) is called an existence proof. There are several ways to prove a theorem of this type. 
Sometimes an existence proof of 4x P (x) can be given by finding an element a, called a witness, 
such that P(a) is true. This type of existence proof is called constructive. It is also possible 
to give an existence proof that is nonconstructive; that is, we do not find an element a such 
that P (a) is true, but rather prove that 4x P(x) is true in some other way. One common method 
of giving a nonconstructive existence proof is to use proof by contradiction and show that the 
negation of the existential quantification implies a contradiction. The concept of a constructive 
existence proof is illustrated by Example 10 and the concept of a nonconstructive existence 
proof is illustrated by Example 11. 


A Constructive Existence Proof Show that there is a positive integer that can be written as 
the sum of cubes of positive integers in two different ways. 


Solution: After considerable computation (such as a computer search) we find that 


1729 = 10° + 99 = 1234 13. 
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Because we have displayed a positive integer that can be written as the sum of cubes in two 
different ways, we are done. 

There is an interesting story pertaining to this example. The English mathematician G. H. 
Hardy, when visiting the ailing Indian prodigy Ramanujan in the hospital, remarked that 1729, 
the number of the cab he took, was rather dull. Ramanujan replied “No, it is a very interesting 
number; it is the smallest number expressible as the sum of cubes in two different ways.” 4 


EXAMPLE 11 A Nonconstructive Existence Proof Show that there exist irrational numbers x and y such 
that x” is rational. 


Solution: By Example 10 in Section 1.7 we know that /2 is irrational. Consider the number 
Ja. If it is rational, we have two irrational numbers x and y with x” rational, namely, x = J2 
and y = /2. On the other hand if Ja is irrational, then we can let x = Ja? and y = J2 
so that x” = (/3? v2 = Jy2vD — Jz —wa 


This proof is an example of a nonconstructive existence proof because we have not found 
irrational numbers x and y such that x” is rational. Rather, we have shown that either the pair 


x = V2, y = V2 orthe pair x = Ja? y = V2 have the desired property, but we do not know 
which of these two pairs works! < 


GODFREY HAROLD HARDY (1877-1947) Hardy, born in Cranleigh, Surrey, England, was the older of 
two children of Isaac Hardy and Sophia Hall Hardy. His father was the geography and drawing master at the 
Cranleigh School and also gave singing lessons and played soccer. His mother gave piano lessons and helped 
run a boardinghouse for young students. Hardy’s parents were devoted to their children’s education. Hardy 
demonstrated his numerical ability at the early age of two when he began writing down numbers into the 
millions. He had a private mathematics tutor rather than attending regular classes at the Cranleigh School. He 
moved to Winchester College, a private high school, when he was 13 and was awarded a scholarship. He excelled 
in his studies and demonstrated a strong interest in mathematics. He entered Trinity College, Cambridge, in 
1896 on a scholarship and won several prizes during his time there, graduating in 1899. 

Hardy held the position of lecturer in mathematics at Trinity College at Cambridge University from 1906 to 1919, when he was 
appointed to the Sullivan chair of geometry at Oxford. He had become unhappy with Cambridge over the dismissal of the famous 
philosopher and mathematician Bertrand Russell from Trinity for antiwar activities and did not like a heavy load of administrative 
duties. In 1931 he returned to Cambridge as the Sadleirian professor of pure mathematics, where he remained until his retirement 
in 1942. He was a pure mathematician and held an elitist view of mathematics, hoping that his research could never be applied. 
Ironically, he is perhaps best known as one of the developers of the Hardy—Weinberg law, which predicts patterns of inheritance. 
His work in this area appeared as a letter to the journal Science in which he used simple algebraic ideas to demonstrate errors in 
an article on genetics. Hardy worked primarily in number theory and function theory, exploring such topics as the Riemann zeta 
function, Fourier series, and the distribution of primes. He made many important contributions to many important problems, such 
as Waring’s problem about representing positive integers as sums of kth powers and the problem of representing odd integers as 
sums of three primes. Hardy is also remembered for his collaborations with John E. Littlewood, a colleague at Cambridge, with 
whom he wrote more than 100 papers, and the famous Indian mathematical prodigy Srinivasa Ramanujan. His collaboration with 
Littlewood led to the joke that there were only three important English mathematicians at that time, Hardy, Littlewood, and Hardy— 
Littlewood, although some people thought that Hardy had invented a fictitious person, Littlewood, because Littlewood was seldom 
seen outside Cambridge. Hardy had the wisdom of recognizing Ramanujan’s genius from unconventional but extremely creative 
writings Ramanujan sent him, while other mathematicians failed to see the genius. Hardy brought Ramanujan to Cambridge and 
collaborated on important joint papers, establishing new results on the number of partitions of an integer. Hardy was interested 
in mathematics education, and his book A Course of Pure Mathematics had a profound effect on undergraduate instruction in 
mathematics in the first half of the twentieth century. Hardy also wrote A Mathematician’s Apology, in which he gives his answer 
to the question of whether it is worthwhile to devote one’s life to the study of mathematics. It presents Hardy’s view of what 
mathematics is and what a mathematician does. 

Hardy had a strong interest in sports. He was an avid cricket fan and followed scores closely. One peculiar trait he had was that 
he did not like his picture taken (only five snapshots are known) and disliked mirrors, covering them with towels immediately upon 
entering a hotel room. 
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Nonconstructive existence proofs often are quite subtle, as Example 12 illustrates. 


EXAMPLE12 Chompisa game played by two players. In this game, cookies are laid out on a rectangular grid. 
The cookie in the top left position is poisoned, as shown in Figure (a). The two players take 
turns making moves; at each move, a player is required to eat a remaining cookie, together with 

all cookies to the right and/or below it (see Figure 1(b), for example). The loser is the player 
Links Qs Who has no choice but to eat the poisoned cookie. We ask whether one of the two players has a 
winning strategy. That is, can one of the players always make moves that are guaranteed to lead 

to a win? 


Solution: We will give a nonconstructive existence proof of a winning strategy for the first 
player. That is, we will show that the first player always has a winning strategy without explicitly 
describing the moves this player must follow. 

First, note that the game ends and cannot finish in a draw because with each move at least 
one cookie is eaten, so after no more than m x n moves the game ends, where the initial grid 
is m x n. Now, suppose that the first player begins the game by eating just the cookie in the 
bottom right corner. There are two possibilities, this is the first move of a winning strategy for 
the first player, or the second player can make a move that is the first move of a winning strategy 
for the second player. In this second case, instead of eating just the cookie in the bottom right 
corner, the first player could have made the same move that the second player made as the first 


SRINIVASA RAMANUJAN (1887-1920) The famous mathematical prodigy Ramanujan was born and raised 
in southern India near the city of Madras (now called Chennai). His father was a clerk in a cloth shop. His mother 
contributed to the family income by singing at a local temple. Ramanujan studied at the local English language 
school, displaying his talent and interest for mathematics. At the age of 13 he mastered a textbook used by 
college students. When he was 15, a university student lent him a copy of Synopsis of Pure Mathematics. 
Ramanujan decided to work out the over 6000 results in this book, stated without proof or explanation, writing 
on sheets later collected to form notebooks. He graduated from high school in 1904, winning a scholarship to the 
University of Madras. Enrolling in a fine arts curriculum, he neglected his subjects other than mathematics and 
lost his scholarship. He failed to pass examinations at the university four times from 1904 to 1907, doing well 
only in mathematics. During this time he filled his notebooks with original writings, sometimes rediscovering already published 
work and at other times making new discoveries. 

Without a university degree, it was difficult for Ramanujan to find a decent job. To survive, he had to depend on the goodwill of 
his friends. He tutored students in mathematics, but his unconventional ways of thinking and failure to stick to the syllabus caused 
problems. He was married in 1909 in an arranged marriage to a young woman nine years his junior. Needing to support himself and 
his wife, he moved to Madras and sought a job. He showed his notebooks of mathematical writings to his potential employers, but 
the books bewildered them. However, a professor at the Presidency College recognized his genius and supported him, and in 1912 
he found work as an accounts clerk, earning a small salary. 

Ramanujan continued his mathematical work during this time and published his first paper in 1910 in an Indian journal. He 
realized that his work was beyond that of Indian mathematicians and decided to write to leading English mathematicians. The first 
mathematicians he wrote to turned down his request for help. But in January 1913 he wrote to G. H. Hardy, who was inclined 
to turn Ramanujan down, but the mathematical statements in the letter, although stated without proof, puzzled Hardy. He decided 
to examine them closely with the help of his colleague and collaborator J. E. Littlewood. They decided, after careful study, that 
Ramanujan was probably a genius, because his statements “could only be written down by a mathematician of the highest class; 
they must be true, because if they were not true, no one would have the imagination to invent them.” 

Hardy arranged a scholarship for Ramanujan, bringing him to England in 1914. Hardy personally tutored him in mathematical 
analysis, and they collaborated for five years, proving significant theorems about the number of partitions of integers. During this 
time, Ramanujan made important contributions to number theory and also worked on continued fractions, infinite series, and elliptic 
functions. Ramanujan had amazing insight involving certain types of functions and series, but his purported theorems on prime 
numbers were often wrong, illustrating his vague idea of what constitutes a correct proof. He was one of the youngest members ever 
appointed a Fellow of the Royal Society. Unfortunately, in 1917 Ramanujan became extremely ill. At the time, it was thought that he 
had trouble with the English climate and had contracted tuberculosis. It is now thought that he suffered from a vitamin deficiency, 
brought on by Ramanujan’s strict vegetarianism and shortages in wartime England. He returned to India in 1919, continuing to 
do mathematics even when confined to his bed. He was religious and thought his mathematical talent came from his family deity, 
Namagiri. He considered mathematics and religion to be linked. He said that “‘an equation for me has no meaning unless it expresses 
a thought of God.” His short life came to an end in April 1920, when he was 32 years old. Ramanujan left several notebooks of 
unpublished results. The writings in these notebooks illustrate Ramanujan’s insights but are quite sketchy. Several mathematicians 
have devoted many years of study to explaining and justifying the results in these notebooks. 
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FIGURE 1 (a) Chomp (Top Left Cookie Poisoned). (b) Three Possible Moves. 


move of a winning strategy (and then continued to follow that winning strategy). This would 
guarantee a win for the first player. 

Note that we showed that a winning strategy exists, but we did not specify an actual winning 
strategy. Consequently, the proof is a nonconstructive existence proof. In fact, no one has been 
able to describe a winning strategy for that Chomp that applies for all rectangular grids by 
describing the moves that the first player should follow. However, winning strategies can be 
described for certain special cases, such as when the grid is square and when the grid only has 
two rows of cookies (see Exercises 15 and 16 in Section 5.2). < 


Uniqueness Proofs 


Some theorems assert the existence of a unique element with a particular property. In other 
words, these theorems assert that there is exactly one element with this property. To prove a 
statement of this type we need to show that an element with this property exists and that no 
other element has this property. The two parts of a uniqueness proof are: 


Existence: We show that an element x with the desired property exists. 
Uniqueness: We show that if y 4 x, then y does not have the desired property. 


Equivalently, we can show that if x and y both have the desired property, then x = y. 


Remark: Showing that there is a unique element x such that P(x) is the same as proving the 
statement 4x(P(x) A Vy(y 4x > -=P(y))). 


We illustrate the elements of a uniqueness proof in Example 13. 


EXAMPLE 13 — Show that if a and b are real numbers and a # 0, then there is a unique real number r such that 
ar+b=0. 


Solution: First, note that the real number r = —D/a is a solution of ar + b = 0 because 
a(—b/a) + b = —b +b = 0. Consequently, a real number r exists for which ar + b = 0. This 
is the existence part of the proof. 
Second, suppose that s is areal number such that as + b = 0.Thenar + b = as + b, where 
Extra r = —b/a. Subtracting b from both sides, we find that ar = as. Dividing both sides of this last 
Examples equation by a, which is nonzero, we see that r = s. This means that ifs Ar, thenas +b 40. 
This establishes the uniqueness part of the proof. 
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Proof Strategies 


Finding proofs can be a challenging business. When you are confronted with a statement to 
prove, you should first replace terms by their definitions and then carefully analyze what the 
hypotheses and the conclusion mean. After doing so, you can attempt to prove the result using 
one of the available methods of proof. Generally, if the statement is a conditional statement, 
you should first try a direct proof; if this fails, you can try an indirect proof. If neither of these 
approaches works, you might try a proof by contradiction. 


FORWARD AND BACKWARD REASONING Whichever method you choose, you need 
a starting point for your proof. To begin a direct proof of a conditional statement, you start 
with the premises. Using these premises, together with axioms and known theorems, you can 
construct a proof using a sequence of steps that leads to the conclusion. This type of reasoning, 
called forward reasoning, is the most common type of reasoning used to prove relatively simple 
results. Similarly, with indirect reasoning you can start with the negation of the conclusion and, 
using a sequence of steps, obtain the negation of the premises. 

Unfortunately, forward reasoning is often difficult to use to prove more complicated results, 
because the reasoning needed to reach the desired conclusion may be far from obvious. In such 
cases it may be helpful to use backward reasoning. To reason backward to prove a statement q, 
we find a statement p that we can prove with the property that p — q. (Note that it is not helpful 
to find a statement r that you can prove such that g — r, because it is the fallacy of begging 
the question to conclude from g — r andr that q is true.) Backward reasoning 1s illustrated in 
Examples 14 and 15. 


Given two positive real numbers x and y, their arithmetic mean is (x + y)/2 and their geo- 
metric mean is ,/xy. When we compare the arithmetic and geometric means of pairs of distinct 
positive real numbers, we find that the arithmetic mean is always greater than the geometric 
mean. [For example, when x = 4 and y = 6, we have 5 = (44+ 6)/2 > V4-6= /24.] Can 
we prove that this inequality is always true? 


Solution: To prove that (x + y)/2 > ./xy when x and y are distinct positive real numbers, 
we can work backward. We construct a sequence of equivalent inequalities. The equivalent 
inequalities are 


(x + y)/2 > /xy, 
(x + y)?/4 > xy, 
(x+y)? > 4xy, 
x? 4+ Ixy + y? > Axy, 
a = Deve = 0, 
@=77 SO, 


Because (x — y)* > 0 when x ¥ y, it follows that the final inequality is true. Because all these 
inequalities are equivalent, it follows that (x + y)/2 > ./xy when x 4 y. Once we have carried 
out this backward reasoning, we can easily reverse the steps to construct a proof using forward 
reasoning. We now give this proof. 

Suppose that x and y are distinct positive real numbers. Then (x — y)? > 0 because 
the square of a nonzero real number is positive (see Appendix 1). Because (x — y)? = 
x? —2xy + y’, this implies that x? — 2xy + y? > 0. Adding 4xy to both sides, we obtain 
x? +2xy + y? > 4xy. Because x? + 2xy + y? = (x + y)’, this means that (x + y)? > 4xy. 
Dividing both sides of this equation by 4, we see that (x + y)?/4 > xy. Finally, taking square 
roots of both sides (which preserves the inequality because both sides are positive) yields 
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(x + y)/2 > ./xy. We conclude that if x and y are distinct positive real numbers, then their 
arithmetic mean (x + y)/2 is greater than their geometric mean ./xy. < 


Suppose that two people play a game taking turns removing one, two, or three stones at a time 
from a pile that begins with 15 stones. The person who removes the last stone wins the game. 
Show that the first player can win the game no matter what the second player does. 


Solution: To prove that the first player can always win the game, we work backward. At the 
last step, the first player can win if this player is left with a pile containing one, two, or three 
stones. The second player will be forced to leave one, two, or three stones if this player has to 
remove stones from a pile containing four stones. Consequently, one way for the first person to 
win is to leave four stones for the second player on the next-to-last move. The first person can 
leave four stones when there are five, six, or seven stones left at the beginning of this player’s 
move, which happens when the second player has to remove stones from a pile with eight stones. 
Consequently, to force the second player to leave five, six, or seven stones, the first player should 
leave eight stones for the second player at the second-to-last move for the first player. This means 
that there are nine, ten, or eleven stones when the first player makes this move. Similarly, the 
first player should leave twelve stones when this player makes the first move. We can reverse 
this argument to show that the first player can always make moves so that this player wins the 
game no matter what the second player does. These moves successively leave twelve, eight, and 
four stones for the second player. 


ADAPTING EXISTING PROOFS Anexcellent way to look for possible approaches that can 
be used to prove a statement is to take advantage of existing proofs of similar results. Often 
an existing proof can be adapted to prove other facts. Even when this is not the case, some of 
the ideas used in existing proofs may be helpful. Because existing proofs provide clues for new 
proofs, you should read and understand the proofs you encounter in your studies. This process 
is illustrated in Example 16. 


In Example 10 of Section 1.7 we proved that /2 is irrational. We now conjecture that /3 is 
irrational. Can we adapt the proof in Example 10 in Section 1.7 to show that J3 is irrational? 


Solution: To adapt the proof in Example 10 in Section 1.7, we begin by mimicking the steps in 
that proof, but with /2 replaced with /3. First, we suppose that /3 = d/c where the fraction 
c/d is in lowest terms. Squaring both sides tells us that 3 = c?/d*, so that 3d” = c*. Can we 
use this equation to show that 3 must be a factor of both c and d, similar to how we used the 
equation 2b* = a” in Example 10 in Section 1.7 to show that 2 must be a factor of both a 
and b? (Recall that an integer s is a factor of the integer ¢ if t/s is an integer. An integer is even 
if and only if 2 is a factor of n.) In turns out that we can, but we need some ammunition from 
number theory, which we will develop in Chapter 4. We sketch out the remainder of the proof, 
but leave the justification of these steps until Chapter 4. Because 3 is a factor of c?, it must also 
be a factor of c. Furthermore, because 3 is a factor of c, 9 is a factor of c2, which means that 9 
is a factor of 3d2. This implies that 3 is a factor of d, which means that 3 is a factor of that d. 
This makes 3 a factor of both c and d, which contradicts the assumption that c/d is in lowest 
terms. After we have filled in the justification for these steps, we will have shown that J/3 is 
irrational by adapting the proof that /2 is irrational. Note that this proof can be extended to 
show that ./n is irrational whenever n is a positive integer that is not a perfect square. We leave 
the details of this to Chapter 4. 


A good tip is to look for existing proofs that you might adapt when you are confronted 
with proving a new theorem, particularly when the new theorem seems similar to one you have 
already proved. 
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Looking for Counterexamples 


In Section 1.7 we introduced the use of counterexamples to show that certain statements are 
false. When confronted with a conjecture, you might first try to prove this conjecture, and if 
your attempts are unsuccessful, you might try to find a counterexample, first by looking at 
the simplest, smallest examples. If you cannot find a counterexample, you might again try to 
prove the statement. In any case, looking for counterexamples is an extremely important pursuit, 
which often provides insights into problems. We will illustrate the role of counterexamples in 
Example 17. 


In Example 14 in Section 1.7 we showed that the statement “Every positive integer is the sum of 
two squares of integers” is false by finding a counterexample. That is, there are positive integers 
that cannot be written as the sum of the squares of two integers. Although we cannot write every 
positive integer as the sum of the squares of two integers, maybe we can write every positive 
integer as the sum of the squares of three integers. That is, is the statement “Every positive 
integer is the sum of the squares of three integers” true or false? 


Solution: Because we know that not every positive integer can be written as the sum of two 
squares of integers, we might initially be skeptical that every positive integer can be written as 
the sum of three squares of integers. So, we first look for a counterexample. That is, we can 
show that the statement “Every positive integer is the sum of three squares of integers” is false 
if we can find a particular integer that is not the sum of the squares of three integers. To look 
for a counterexample, we try to write successive positive integers as a sum of three squares. 
We find that 1 = 0? +074 17,2=0?4+17+17,3=174+1?4+17,4=0°+0°4+27,5= 
074+ 174+. 2?,6= 12 + 12+ 2?, but we cannot find a way to write 7 as the sum of three 
squares. To show that there are not three squares that add up to 7, we note that the only possible 
squares we can use are those not exceeding 7, namely, 0, 1, and 4. Because no three terms where 
each term is 0, 1, or 4 add up to 7, it follows that 7 is a counterexample. We conclude that the 
statement “Every positive integer is the sum of the squares of three integers” is false. 

We have shown that not every positive integer is the sum of the squares of three integers. 
The next question to ask is whether every positive integer is the sum of the squares of four 
positive integers. Some experimentation provides evidence that the answer is yes. For example, 
T= P41? 41° 427,25 = 47 42? +274 17, and 87 = 9? + 2? + 1? + 17. It turns out the 
conjecture “Every positive integer is the sum of the squares of four integers” is true. For a proof, 
see [Rol0]. < 


Proof Strategy in Action 


Mathematics is generally taught as if mathematical facts were carved in stone. Mathematics 
texts (including the bulk of this book) formally present theorems and their proofs. Such presen- 
tations do not convey the discovery process in mathematics. This process begins with exploring 
concepts and examples, asking questions, formulating conjectures, and attempting to settle these 
conjectures either by proof or by counterexample. These are the day-to-day activities of math- 
ematicians. Believe it or not, the material taught in textbooks was originally developed in this 
way. 

People formulate conjectures on the basis of many types of possible evidence. The exam- 
ination of special cases can lead to a conjecture, as can the identification of possible patterns. 
Altering the hypotheses and conclusions of known theorems also can lead to plausible conjec- 
tures. At other times, conjectures are made based on intuition or a belief that a result holds. 
No matter how a conjecture was made, once it has been formulated, the goal is to prove or 
disprove it. When mathematicians believe that a conjecture may be true, they try to find a proof. 
If they cannot find a proof, they may look for a counterexample. When they cannot find a coun- 
terexample, they may switch gears and once again try to prove the conjecture. Although many 
conjectures are quickly settled, a few conjectures resist attack for hundreds of years and lead to 
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FIGURE 3 
FIGURE 2. The Standard Checkerboard. Two Dominoes. 


the development of new parts of mathematics. We will mention a few famous conjectures later 
in this section. 


Tilings 


We can illustrate aspects of proof strategy through a brief study of tilings of checkerboards. 
Looking at tilings of checkerboards is a fruitful way to quickly discover many different results 
and construct their proofs using a variety of proof methods. There are almost an endless number 
of conjectures that can be made and studied in this area too. To begin, we need to define some 
terms. A checkerboard is a rectangle divided into squares of the same size by horizontal and 
vertical lines. The game of checkers is played on a board with 8 rows and 8 columns; this 
board is called the standard checkerboard and is shown in Figure 2. In this section we use the 
term board to refer to a checkerboard of any rectangular size as well as parts of checkerboards 
obtained by removing one or more squares. A domino is a rectangular piece that is one square 
by two squares, as shown in Figure 3. We say that a board is tiled by dominoes when all its 
squares are covered with no overlapping dominoes and no dominoes overhanging the board. We 
now develop some results about tiling boards using dominoes. 


Can we tile the standard checkerboard using dominoes? 


Solution: We can find many ways to tile the standard checkerboard using dominoes. For example, 
we can tile it by placing 32 dominoes horizontally, as shown in Figure 4. The existence of one 
such tiling completes a constructive existence proof. Of course, there are a large number of other 
ways to do this tiling. We can place 32 dominoes vertically on the board or we can place some 
tiles vertically and some horizontally. But for a constructive existence proof we needed to find 
just one such tiling. 


Can we tile a board obtained by removing one of the four corner squares of a standard checker- 
board? 


Solution: To answer this question, note that a standard checkerboard has 64 squares, so removing 
a square produces a board with 63 squares. Now suppose that we could tile a board obtained 
from the standard checkerboard by removing a corner square. The board has an even number of 
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FIGURE 4 Tiling the Standard Checkerboard. FIGURE 5 The Standard Checkerboard 
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squares because each domino covers two squares and no two dominoes overlap and no dominoes 
overhang the board. Consequently, we can prove by contradiction that a standard checkerboard 
with one square removed cannot be tiled using dominoes because such a board has an odd 
number of squares. < 


We now consider a trickier situation. 


Can we tile the board obtained by deleting the upper left and lower right corner squares of a 
standard checkerboard, shown in Figure 5? 


Solution: A board obtained by deleting two squares of a standard checkerboard contains 
64 — 2 = 62 squares. Because 62 is even, we cannot quickly rule out the existence of a tiling of 
the standard checkerboard with its upper left and lower right squares removed, unlike Example 
19, where we ruled out the existence of a tiling of the standard checkerboard with one corner 
square removed. Trying to construct a tiling of this board by successively placing dominoes 
might be a first approach, as the reader should attempt. However, no matter how much we try, 
we cannot find such a tiling. Because our efforts do not produce a tiling, we are led to conjecture 
that no tiling exists. 

We might try to prove that no tiling exists by showing that we reach a dead end however 
we successively place dominoes on the board. To construct such a proof, we would have to 
consider all possible cases that arise as we run through all possible choices of successively 
placing dominoes. For example, we have two choices for covering the square in the second 
column of the first row, next to the removed top left corner. We could cover it with a horizontally 
placed tile or a vertically placed tile. Each of these two choices leads to further choices, and so 
on. It does not take long to see that this is not a fruitful plan of attack for a person, although a 
computer could be used to complete such a proof by exhaustion. (Exercise 45 asks you to supply 
such a proof to show that a 4 x 4 checkerboard with opposite corners removed cannot be tiled.) 

We need another approach. Perhaps there is an easier way to prove there is no tiling of a 
standard checkerboard with two opposite corners removed. As with many proofs, a key obser- 
vation can help. We color the squares of this checkerboard using alternating white and black 
squares, as in Figure 2. Observe that a domino in a tiling of such a board covers one white square 
and one black square. Next, note that this board has unequal numbers of white square and black 
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squares. We can use these observations to prove by contradiction that a standard checkerboard 
with opposite corners removed cannot be tiled using dominoes. We now present such a proof. 


Proof: Suppose we can use dominoes to tile a standard checkerboard with opposite corners 
removed. Note that the standard checkerboard with opposite corners removed contains 64 — 2 = 
62 squares. The tiling would use 62/2 = 31 dominoes. Note that each domino in this tiling covers 
one white and one black square. Consequently, the tiling covers 31 white squares and 31 black 
squares. However, when we remove two opposite corner squares, either 32 of the remaining 
squares are white and 30 are black or else 30 are white and 32 are black. This contradicts the 
assumption that we can use dominoes to cover a standard checkerboard with opposite corners 
removed, completing the proof. < 


We can use other types of pieces besides dominoes in tilings. Instead of dominoes we can 
study tilings that use identically shaped pieces constructed from congruent squares that are 
connected along their edges. Such pieces are called polyominoes, a term coined in 1953 by the 
mathematician Solomon Golomb, the author of an entertaining book about them [Go94]. We 
will consider two polyominoes with the same number of squares the same if we can rotate and/or 
flip one of the polyominoes to get the other one. For example, there are two types of triominoes 
(see Figure 6), which are polyominoes made up of three squares connected by their sides. One 
type of triomino, the straight triomino, has three horizontally connected squares; the other 
type, right triominoes, resembles the letter L in shape, flipped and/or rotated, if necessary. We 
will study the tilings of a checkerboard by straight triominoes here; we will study tilings by 
right triominoes in Section 5.1. 


Can you use straight triominoes to tile a standard checkerboard? 


Solution: The standard checkerboard contains 64 squares and each triomino covers three 
squares. Consequently, if triominoes tile a board, the number of squares of the board must be 
a multiple of 3. Because 64 is not a multiple of 3, triominoes cannot be used to cover an 8 x 8 
checkerboard. < 


In Example 22, we consider the problem of using straight triominoes to tile a standard 
checkerboard with one corner missing. 


Can we use straight triominoes to tile a standard checkerboard with one of its four corners 
removed? An 8 x 8 checkerboard with one corner removed contains 64 — 1 = 63 squares. Any 
tiling by straight triominoes of one of these four boards uses 63/3 = 21 triominoes. However, 
when we experiment, we cannot find a tiling of one of these boards using straight triominoes. 
A proof by exhaustion does not appear promising. Can we adapt our proof from Example 20 to 
prove that no such tiling exists? 


Solution: We will color the squares of the checkerboard in an attempt to adapt the proof by 
contradiction we gave in Example 20 of the impossibility of using dominoes to tile a standard 
checkerboard with opposite corners removed. Because we are using straight triominoes rather 
than dominoes, we color the squares using three colors rather than two colors, as shown in 
Figure 7. Note that there are 21 blue squares, 21 black squares, and 22 white squares in this 
coloring. Next, we make the crucial observation that when a straight triomino covers three 
squares of the checkerboard, it covers one blue square, one black square, and one white square. 
Next, note that each of the three colors appears in a corner square. Thus without loss of generality, 
we may assume that we have rotated the coloring so that the missing square is colored blue. 
Therefore, we assume that the remaining board contains 20 blue squares, 21 black squares, and 
22 white squares. 

If we could tile this board using straight triominoes, then we would use 63/3 = 21 straight 
triominoes. These triominoes would cover 21 blue squares, 21 black squares, and 21 white 
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FIGURE 7 Coloring the Squares of the Standard Checkerboard 
with Three Colors. 


squares. This contradicts the fact that this board contains 20 blue squares, 21 black squares, and 
22 white squares. Therefore we cannot tile this board using straight triominoes. 


The Role of Open Problems 


Many advances in mathematics have been made by people trying to solve famous unsolved 
problems. In the past 20 years, many unsolved problems have finally been resolved, such as the 
proof of a conjecture in number theory made more than 300 years ago. This conjecture asserts 
the truth of the statement known as Fermat’s last theorem. 


FERMAT’S LAST THEOREM The equation 


x” ai y” = ge 


has no solutions in integers x, y, and z with xyz 4 0 whenever n is an integer with n > 2. 


Remark: The equation x* + y? = 2” has infinitely many solutions in integers x, y, and z; these 
solutions are called Pythagorean triples and correspond to the lengths of the sides of right 
triangles with integer lengths. See Exercise 32. 


This problem has a fascinating history. In the seventeenth century, Fermat jotted in the 
margin of his copy of the works of Diophantus that he had a “wondrous proof” that there are no 
integer solutions of x” + y” = z” when n is an integer greater than 2 with xyz 4 0. However, 
he never published a proof (Fermat published almost nothing), and no proof could be found in 
the papers he left when he died. Mathematicians looked for a proof for three centuries without 
success, although many people were convinced that a relatively simple proof could be found. 
(Proofs of special cases were found, such as the proof of the case when n = 3 by Euler and the 
proof of the n = 4 case by Fermat himself.) Over the years, several established mathematicians 
thought that they had proved this theorem. In the nineteenth century, one of these failed attempts 
led to the development of the part of number theory called algebraic number theory. A correct 
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proof, requiring hundreds of pages of advanced mathematics, was not found until the 1990s, 
when Andrew Wiles used recently developed ideas from a sophisticated area of number theory 
called the theory of elliptic curves to prove Fermat’s last theorem. Wiles’s quest to find a 
proof of Fermat’s last theorem using this powerful theory, described in a program in the Nova 
series on public television, took close to ten years! Moreover, his proof was based on major 
contributions of many mathematicians. (The interested reader should consult [Ro10] for more 
information about Fermat’s last theorem and for additional references concerning this problem 
and its resolution.) 

We now state an open problem that is simple to describe, but that seems quite difficult to 
resolve. 


The 3x +1 Conjecture Let T be the transformation that sends an even integer x to x/2 and 
an odd integer x to 3x + 1. A famous conjecture, sometimes known as the 3x + 1 conjec- 
ture, states that for all positive integers x, when we repeatedly apply the transformation T, 
we will eventually reach the integer 1. For example, starting with x = 13, we find T(13) = 
3-13+1=40, T(40) = 40/2 = 20, T(20) = 20/2= 10, TU0)=10/2=5, T(5)= 
3-5+1= 16, T(16) = 8, T(8) =4, T(4) = 2, and T(2) = 1. The 3x + | conjecture has 
been verified using computers for all integers x up to 5.6 - 10/3. 

The 3x + | conjecture has an interesting history and has attracted the attention of mathe- 
maticians since the 1950s. The conjecture has been raised many times and goes by many other 
names, including the Collatz problem, Hasse’s algorithm, Ulam’s problem, the Syracuse prob- 
lem, and Kakutani’s problem. Many mathematicians have been diverted from their work to spend 
time attacking this conjecture. This led to the joke that this problem was part of a conspiracy 
to slow down American mathematical research. See the article by Jeffrey Lagarias [Lal0] for a 
fascinating discussion of this problem and the results that have been found by mathematicians 
attacking it. 


In Chapter 4 we will describe additional open questions about prime numbers. Students 
already familiar with the basic notions about primes might want to explore Section 4.3, where 
these open questions are discussed. We will mention other important open questions throughout 
the book. 


Additional Proof Methods 


In this chapter we introduced the basic methods used in proofs. We also described how to leverage 
these methods to prove a variety of results. We will use these proof methods in all subsequent 
chapters. In particular, we will use them in Chapters 2, 3, and 4 to prove results about sets, 
functions, algorithms, and number theory and in Chapters 9, 10, and 11 to prove results in graph 
theory. Among the theorems we will prove is the famous halting theorem which states that there 
is a problem that cannot be solved using any procedure. However, there are many important 
proof methods besides those we have covered. We will introduce some of these methods later 
in this book. In particular, in Section 5.1 we will discuss mathematical induction, which is an 
extremely useful method for proving statements of the form Vn P (n), where the domain consists 
of all positive integers. In Section 5.3 we will introduce structural induction, which can be used 
to prove results about recursively defined sets. We will use the Cantor diagonalization method, 
which can be used to prove results about the size of infinite sets, in Section 2.5. In Chapter 6 
we will introduce the notion of combinatorial proofs, which can be used to prove results by 
counting arguments. The reader should note that entire books have been devoted to the activities 
discussed in this section, including many excellent works by George Pélya ([Po61], [Po71], 
[Po90]). 

Finally, note that we have not given a procedure that can be used for proving theorems in 
mathematics. It is a deep theorem of mathematical logic that there is no such procedure. 
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Exercises 


1. Prove that n? + 1 > 2” when 2 is a positive integer with 


1l<n<4. 


2. Prove that there are no positive perfect cubes less than 


3. 


10. 


11 


1000 that are the sum of the cubes of two positive integers. 


Prove that if x and y are real numbers, then max(x, y) + 
min(x, y) = x+y. [Hint: Use a proof by cases, with 
the two cases corresponding to x > y andx < y, respec- 
tively.] 

Use a proof by cases to show that min(a, min(b, c)) = 
min(min(a, b), c) whenever a, b, and c are real numbers. 


. Prove using the notion of without loss of generality 


12. 


13 


14. 


15. 


16 


17 


that min(x, y) = (x + y — |x — y|)/2 and max(x, y) = 
(x + y + |x — y|)/2 whenever x and y are real numbers. 
Prove using the notion of without loss of generality that 
5x + 5y is an odd integer when x and y are integers of 
opposite parity. 

Prove the triangle inequality, which states that if x and 
y are real numbers, then |x| + |y| > |x + y| (where |x| 
represents the absolute value of x, which equals x if x > 0 
and equals —x if x < 0). 


Prove that there is a positive integer that equals the sum 
of the positive integers not exceeding it. Is your proof 
constructive or nonconstructive? 


Prove that there are 100 consecutive positive integers that 
are not perfect squares. Is your proof constructive or non- 
constructive? 


Prove that either 2 - 10°" + 15 or 2 - 10°? + 16 is nota 
perfect square. Is your proof constructive or nonconstruc- 
tive? 

Prove that there exists a pair of consecutive integers such 
that one of these integers is a perfect square and the other 
is a perfect cube. 

Show that the product of two of the numbers 65100 — 
92001 A 3177 791212 _ 92399 zi 92001 and 7.44493 _ 
58192 4 71777 is nonnegative. Is your proof constructive 
or nonconstructive? [Hint: Do not try to evaluate these 
numbers! ] 

Prove or disprove that there is a rational number x and an 
irrational number y such that x” is irrational. 

Prove or disprove that if a and b are rational numbers, 
then a? is also rational. 

Show that each of these statements can be used to ex- 
press the fact that there is a unique element x such that 
P(x) is true. [Note that we can also write this statement 
as d!lx P(x).] 

a) AxVy(P(y) ox =y) 

b) ax P(x) A WxVy(P(x) A PQ) > xX = y) 

ce) Ax(P(x) AVy(P(y) > x = y)) 

Show that if a, b, and c are real numbers anda # 0, then 
there is a unique solution of the equation ax +b=c. 
Suppose that a and b are odd integers with a 4 b. Show 
there is a unique integer c such that |a — c| = |b —c]. 


18. Show that if r is an irrational number, there is a unique 


19 


20 


21 


22 


integer 1 such that the distance between r and n is less 
than 1/2. 

Show that if n is an odd integer, then there is a unique 
integer k such that n is the sum of k — 2 andk + 3. 
Prove that given a real number x there exist unique num- 
bers n and € such that x =n +, n is an integer, and 
O<e<l. 


Prove that given a real number x there exist unique num- 
bers n and € such that x =n — €, n is an integer, and 
O<e<l. 


Use forward reasoning to show that if x is a nonzero real 
number, then x4] [x2 > 2. [Hint: Start with the in- 
equality (x — 1/x)? > 0 which holds for all nonzero real 
numbers x.] 


23. The harmonic mean of two real numbers x and y equals 


2xy/(x + y).By computing the harmonic and geometric 
means of different pairs of positive real numbers, formu- 
late a conjecture about their relative sizes and prove your 
conjecture. 


24. The quadratic mean of two real numbers x and y 


equals ,/(x2 + y2)/2. By computing the arithmetic and 
quadratic means of different pairs of positive real num- 
bers, formulate a conjecture about their relative sizes and 
prove your conjecture. 


*25, Write the numbers 1, 2,..., 2” on a blackboard, where 


*26 


27 


28 


29 


30 


31 


32 


n is an odd integer. Pick any two of the numbers, j and 
k, write |j — k| on the board and erase j and k. Continue 
this process until only one integer is written on the board. 
Prove that this integer must be odd. 


Suppose that five ones and four zeros are arranged around 
a circle. Between any two equal bits you insert a 0 and 
between any two unequal bits you insert a | to produce 
nine new bits. Then you erase the nine original bits. Show 
that when you iterate this procedure, you can never get 
nine zeros. [Hint: Work backward, assuming that you did 
end up with nine zeros.] 


Formulate a conjecture about the decimal digits that ap- 
pear as the final decimal digit of the fourth power of an 
integer. Prove your conjecture using a proof by cases. 


Formulate a conjecture about the final two decimal digits 
of the square of an integer. Prove your conjecture using a 
proof by cases. 


Prove that there is no positive integer n such that n? + 
3 
n°? = 100. 


Prove that there are no solutions in integers x and y to the 
equation 2x? + 5y* = 14. 
Prove that there are no solutions in positive integers x and 


y to the equation x* + y+ = 625. 

Prove that there are infinitely many solutions in posi- 
tive integers x, y, and z to the equation x” + y? = 2’. 
[Hint: Let x = m2 — n2, y = 2mn, and z= m2 + n2, 
where m and n are integers.] 


33. 


34. 
35. 
36. 


* 37. 


38. 


39. 


40. 


41. 


42. 


43. 


44. 


45. 


Adapt the proof in Example 4 in Section 1.7 to prove that 
if n = abc, where a, b, and c are positive integers, then 


a<Jn,b < Yn, ore < Yn. 
Prove that </2 is irrational. 


Prove that between every two rational numbers there is 
an irrational number. 


Prove that between every rational number and every irra- 
tional number there is an irrational number. 


Let S=x1y1 + x2y2 +++++XnYn, where x1, x2,..., 
Xn and yj, y2,..., Yp are orderings of two different se- 
quences of positive real numbers, each containing n ele- 
ments. 


a) Show that S takes its maximum value over all order- 
ings of the two sequences when both sequences are 
sorted (so that the elements in each sequence are in 
nondecreasing order). 

b) Show that S$ takes its minimum value over all order- 
ings of the two sequences when one sequence is sorted 
into nondecreasing order and the other is sorted into 
nonincreasing order. 


Prove or disprove that if you have an 8-gallon jug of wa- 
ter and two empty jugs with capacities of 5 gallons and 3 
gallons, respectively, then you can measure 4 gallons by 
successively pouring some of or all of the water in a jug 
into another jug. 


Verify the 3x + 1 conjecture for these integers. 

a) 6 b) 7 c) 17 d) 21 

Verify the 3x + 1 conjecture for these integers. 

a) 16 b) 11 c) 35 d) 113 

Prove or disprove that you can use dominoes to tile 
the standard checkerboard with two adjacent corners re- 
moved (that is, corners that are not opposite). 

Prove or disprove that you can use dominoes to tile a 
standard checkerboard with all four corners removed. 
Prove that you can use dominoes to tile a rectangular 
checkerboard with an even number of squares. 

Prove or disprove that you can use dominoes to tile a 
5 x 5 checkerboard with three corners removed. 

Use a proof by exhaustion to show that a tiling using 
dominoes of a 4 x 4 checkerboard with opposite corners 
removed does not exist. [Hint: First show that you can 
assume that the squares in the upper left and lower right 
corners are removed. Number the squares of the original 


Key Terms and Results 


TERMS 


proposition: a statement that is true or false 


propositional variable: a variable that represents a proposi- 
tion 


truth value: true or false 


— p (negation of p): the proposition with truth value opposite 
to the truth value of p 
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checkerboard from 1 to 16, starting in the first row, mov- 
ing right in this row, then starting in the leftmost square 
in the second row and moving right, and so on. Remove 
squares | and 16. To begin the proof, note that square 2 is 
covered either by a domino laid horizontally, which cov- 
ers squares 2 and 3, or vertically, which covers squares 2 
and 6. Consider each of these cases separately, and work 
through all the subcases that arise.] 


*46. Prove that when a white square and a black square are 


removed from an 8 x 8 checkerboard (colored as in the 
text) you can tile the remaining squares of the checker- 
board using dominoes. [Hint: Show that when one black 
and one white square are removed, each part of the parti- 
tion of the remaining cells formed by inserting the barriers 
shown in the figure can be covered by dominoes. ] 


47. Show that by removing two white squares and two black 


squares from an 8 x 8 checkerboard (colored as in the 
text) you can make it impossible to tile the remaining 
squares using dominoes. 


* 48. Find all squares, if they exist, on an 8 x 8 checkerboard 


such that the board obtained by removing one of these 
square can be tiled using straight triominoes. [Hint: First 
use arguments based on coloring and rotations to elimi- 
nate as many squares as possible from consideration. ] 


* 49. a) Draw each of the five different tetrominoes, where a 


tetromino is a polyomino consisting of four squares. 

b) Foreach of the five different tetrominoes, prove or dis- 
prove that you can tile a standard checkerboard using 
these tetrominoes. 


*50. Prove or disprove that you can tile a 10 x 10 checker- 


board using straight tetrominoes. 


logical operators: operators used to combine propositions 


compound proposition: a proposition constructed by combin- 


ing propositions using logical operators 


truth table: a table displaying all possible truth values of 


propositions 


PV q (disjunction of p and q): the proposition “p or q,” which 


is true if and only if at least one of p and gq is true 


110 1/The Foundations: Logic and Proofs 


P A q (conjunction of p and q): the proposition “p and q,” 
which is true if and only if both p and q are true 

P © q (exclusive or of p and q): the proposition “p XOR q,” 
which is true when exactly one of p and q is true 

P — q (& implies q): the proposition “if p, then g,” which is 
false if and only if p is true and q is false 

converse of p — q: the conditional statement g > p 

contrapositive of p — q: the conditional statement -qg — =p 


inverse of p — q: the conditional statement —p > —=q 

p @ q (biconditional): the proposition “p if and only if g,” 
which is true if and only if p and q have the same truth 
value 

bit: either a 0 or a 1 

Boolean variable: a variable that has a value of 0 or 1 

bit operation: an operation on a bit or bits 

bit string: a list of bits 

bitwise operations: operations on bit strings that operate on 
each bit in one string and the corresponding bit in the other 
string 

logic gate: a logic element that performs a logical operation 
on one or more bits to produce an output bit 

logic circuit: a switching circuit made up of logic gates that 
produces one or more output bits 

tautology: a compound proposition that is always true 

contradiction: a compound proposition that is always false 

contingency: a compound proposition that is sometimes true 
and sometimes false 

consistent compound propositions: compound propositions 
for which there is an assignment of truth values to the vari- 
ables that makes all these propositions true 

satisfiable compound proposition: a compound proposition 
for which there is an assignment of truth values to its vari- 
ables that makes it true 

logically equivalent compound propositions: compound 
propositions that always have the same truth values 

predicate: part of a sentence that attributes a property to the 
subject 

propositional function: a statement containing one or more 
variables that becomes a proposition when each of its vari- 
ables is assigned a value or is bound by a quantifier 

domain (or universe) of discourse: the values a variable in a 
propositional function may take 

dx P(x) (existential quantification of P(x)): the proposition 
that is true if and only if there exists an x in the domain 
such that P(x) is true 

VxP(x) (universal quantification of P(x)): the proposition 
that is true if and only if P(x) is true for every x in the 
domain 

logically equivalent expressions: expressions that have the 
same truth value no matter which propositional functions 
and domains are used 

free variable: a variable not bound in a propositional function 


bound variable: a variable that is quantified 

scope of a quantifier: portion of a statement where the quan- 
tifier binds its variable 

argument: a sequence of statements 


argument form: a sequence of compound propositions involv- 
ing propositional variables 

premise: a statement, in an argument, or argument form, other 
than the final one 

conclusion: the final statement in an argument or argument 
form 

valid argument form: a sequence of compound propositions 
involving propositional variables where the truth of all the 
premises implies the truth of the conclusion 

valid argument: an argument with a valid argument form 

rule of inference: a valid argument form that can be used in 
the demonstration that arguments are valid 

fallacy: an invalid argument form often used incorrectly as a 
tule of inference (or sometimes, more generally, an incor- 
rect argument) 

circular reasoning or begging the question: reasoning where 
one or more steps are based on the truth of the statement 
being proved 

theorem: a mathematical assertion that can be shown to be 
true 

conjecture: a mathematical assertion proposed to be true, but 
that has not been proved 

proof: a demonstration that a theorem is true 

axiom: a statement that is assumed to be true and that can be 
used as a basis for proving theorems 

lemma: a theorem used to prove other theorems 

corollary: a proposition that can be proved as a consequence 
of a theorem that has just been proved 

vacuous proof: a proof that p — q is true based on the fact 
that p is false 

trivial proof: a proof that p — q is true based on the fact that 
q is true 

direct proof: a proof that p — q is true that proceeds by show- 
ing that g must be true when p is true 

proof by contraposition: a proof that p — q is true that pro- 
ceeds by showing that p must be false when gq is false 

proof by contradiction: a proof that p is true based on the 
truth of the conditional statement =p — q, where g is a 
contradiction 

exhaustive proof: a proof that establishes a result by checking 
a list of all possible cases 

proof by cases: a proof broken into separate cases, where these 
cases cover all possibilities 

without loss of generality: an assumption in a proof that makes 
it possible to prove a theorem by reducing the number of 
cases to consider in the proof 

counterexample: an element x such that P(x) is false 

constructive existence proof: a proof that an element with a 
specified property exists that explicitly finds such an ele- 
ment 

nonconstructive existence proof: a proof that an element with 
a specified property exists that does not explicitly find such 
an element 

rational number: a number that can be expressed as the ratio 
of two integers p and q such that g #0 

uniqueness proof: a proof that there is exactly one element 
satisfying a specified property 


RESULTS 


The logical equivalences given in Tables 6, 7, and 8 in Sec- 


tion 1.3. 


Review Questions 


1. 


a) Define the negation of a proposition. 
b) What is the negation of “This is a boring course”? 


. a) Define (using truth tables) the disjunction, conjunc- 


tion, exclusive or, conditional, and biconditional of 
the propositions p and q. 

b) What are the disjunction, conjunction, exclusive or, 
conditional, and biconditional of the propositions “T’Il 
go to the movies tonight” and “I'll finish my discrete 
mathematics homework”? 


. a) Describe at least five different ways to write the con- 


ditional statement p — q in English. 

b) Define the converse and contrapositive of a conditional 
statement. 

c) State the converse and the contrapositive of the con- 
ditional statement “If it is sunny tomorrow, then I will 
go for a walk in the woods.” 


. a) What does it mean for two propositions to be logically 


equivalent? 

b) Describe the different ways to show that two com- 
pound propositions are logically equivalent. 

c) Show inat least two different ways that the compound 
propositions sp Vv (r > 7q) and =p Vv -q V —r are 
equivalent. 


. (Depends on the Exercise Set in Section 1.3) 


a) Given a truth table, explain how to use disjunctive nor- 
mal form to construct a compound proposition with 
this truth table. 

b) Explain why part (a) shows that the operators A, V, 
and — are functionally complete. 

c) Is there an operator such that the set containing just 
this operator is functionally complete? 


. What are the universal and existential quantifications of 


a predicate P(x)? What are their negations? 


. a) What is the difference between the quantification 


AxVyP(x, y) and Vyax P(x, y), where P(x, y) is a 
predicate? 


Supplementary Exercises 


1. 


Let p be the proposition “I will do every exercise in 
this book” and q be the proposition “I will get an “A” 
in this course.” Express each of these as a combination of 
p and q. 


a) I will get an “A” in this course only if I do every exer- 
cise in this book. 
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De Morgan’s laws for quantifiers. 
Rules of inference for propositional calculus. 
Rules of inference for quantified statements. 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


b) Give an example of a predicate P(x, y) such that 
dxVy P(x, y) and Vydx P(x, y) have different truth 
values. 


. Describe what is meant by a valid argument in proposi- 


tional logic and show that the argument “If the earth is 
flat, then you can sail off the edge of the earth,” “You can- 
not sail off the edge of the earth,” therefore, “The earth is 
not flat” is a valid argument. 


. Use rules of inference to show that if the premises “All 


zebras have stripes” and “Mark is a zebra” are true, then 
the conclusion “Mark has stripes” is true. 


a) Describe what is meant by a direct proof, a proof by 
contraposition, and a proof by contradiction of a con- 
ditional statement p > q. 

b) Give a direct proof, a proof by contraposition and a 
proof by contradiction of the statement: “If n is even, 
then n + 4 is even.” 


a) Describe a way to prove the biconditional p + q. 
b) Prove the statement: “The integer 3” + 2 is odd if and 
only if the integer 9n + 5 is even, where n is an inte- 


” 


ger. 


To prove that the statements p1, p2, p3, and p4 are equiva- 
lent, is it sufficient to show that the conditional statements 
Pa —> p2, p3 > pi, and py — pz are valid? If not, pro- 
vide another collection of conditional statements that can 
be used to show that the four statements are equivalent. 


a) Suppose that a statement of the form Vx P (x) is false. 
How can this be proved? 

b) Show that the statement “For every positive integer n, 
n2 > 2n” is false. 

What is the difference between a constructive and non- 

constructive existence proof? Give an example of each. 

What are the elements of a proof that there is a unique 

element x such that P(x), where P(x) is a propositional 

function? 

Explain how a proof by cases can be used to prove a result 

about absolute values, such as the fact that |xy| = |x||y| 

for all real numbers x and y. 


b) I will get an “A” in this course and I will do every 
exercise in this book. 


c) Either I will not get an “A” in this course or I will not 
do every exercise in this book. 


d) For me to get an “A” in this course it is necessary and 
sufficient that I do every exercise in this book. 
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. Find the truth table of the compound proposition (p V 


q) > (pA). 


. Show that these compound propositions are tautologies. 


a) (-qA (p> 4q))> 7p 
b) (pV q)A7p)> 4 


. Give the converse, the contrapositive, and the inverse of 


these conditional statements. 


a) If it rains today, then I will drive to work. 
b) If |x| = x, then x > 0. 
c) If n is greater than 3, then n? is greater than 9. 


. Given a conditional statement p — gq, find the converse 


of its inverse, the converse of its converse, and the con- 
verse of its contrapositive. 


. Given a conditional statement p — q, find the inverse of 


its inverse, the inverse of its converse, and the inverse of 
its contrapositive. 


. Find acompound proposition involving the propositional 


variables p, g,r, and s that is true when exactly three of 
these propositional variables are true and is false other- 
wise. 


. Show that these statements are inconsistent: “If Sergei 


takes the job offer then he will get a signing bonus.” “If 
Sergei takes the job offer, then he will receive a higher 
salary.” “If Sergei gets a signing bonus, then he will not 
receive a higher salary.” “Sergei takes the job offer.” 


. Show that these statements are inconsistent: “If Miranda 


does not take a course in discrete mathematics, then she 
will not graduate.” “If Miranda does not graduate, then 
she is not qualified for the job.” “If Miranda reads this 
book, then she is qualified for the job.” “Miranda does 
not take a course in discrete mathematics but she reads 
this book.” 


Teachers in the Middle Ages supposedly tested the realtime 
propositional logic ability of a student via a technique known 
as an obligato game. In an obligato game, a number of rounds 
is set and in each round the teacher gives the student succes- 
sive assertions that the student must either accept or reject as 
they are given. When the student accepts an assertion, it is 
added as a commitment; when the student rejects an assertion 
its negation is added as a commitment. The student passes 
the test if the consistency of all commitments is maintained 
throughout the test. 


10. 


11. 


12. 


Suppose that in a three-round obligato game, the teacher 
first gives the student the proposition p — gq, then the 
proposition —=(p V r) V q, and finally the proposition q. 
For which of the eight possible sequences of three answers 
will the student pass the test? 


Suppose that in a four-round obligato game, the teacher 
first gives the student the proposition =(p > (q A7r)), 
then the proposition p V —q, then the proposition —r, and 
finally, the proposition (p Ar) V (q — p). For which of 
the 16 possible sequences of four answers will the student 
pass the test? 


Explain why every obligato game has a winning strategy. 


Exercises 13 and 14 are set on the island of knights and knaves 
described in Example 7 in Section 1.2. 


13. 


14. 


15. 


16. 


17. 


18. 


*19, 


20. 


21. 


22. 


Suppose that you meet three people Aaron, Bohan, and 
Crystal. Can you determine what Aaron, Bohan, and Crys- 
tal are if Aaron says “All of us are knaves” and Bohan says 
“Exactly one of us is a knave.”? 


Suppose that you meet three people, Anita, Boris, and 
Carmen. What are Anita, Boris, and Carmen if Anita says 
“Tam a knave and Boris is a knight” and Boris says “Ex- 
actly one of the three of us is a knight’? 


(Adapted from [Sm78]) Suppose that on an island there 
are three types of people, knights, knaves, and normals 
(also known as spies). Knights always tell the truth, 
knaves always lie, and normals sometimes lie and some- 
times tell the truth. Detectives questioned three inhabi- 
tants of the island—Amy, Brenda, and Claire—as part 
of the investigation of a crime. The detectives knew that 
one of the three committed the crime, but not which one. 
They also knew that the criminal was a knight, and that the 
other two were not. Additionally, the detectives recorded 
these statements: Amy: “I am innocent.” Brenda: “What 
Amy says is true.” Claire: “Brenda is not a normal.” Af- 
ter analyzing their information, the detectives positively 
identified the guilty party. Who was it? 


Show that if S is a proposition, where S is the conditional 
statement “If S is true, then unicorns live,’ then “Uni- 
corns live” is true. Show that it follows that S cannot be a 
proposition. (This paradox is known as Léb’s paradox.) 


Show that the argument with premises “The tooth fairy is a 
real person” and “The tooth fairy is not a real person” and 
conclusion “You can find gold at the end of the rainbow” 
is a valid argument. Does this show that the conclusion is 
true? 


Suppose that the truth value of the proposition p; is T 

whenever i is an odd positive integer and is F when- 

ever i is an even positive integer. Find the truth values 

of Vit (pi A pi+1) and Aj (pi V pi41)- 

Model 16 x 16 Sudoku puzzles (with 4 x 4 blocks) as 

satisfiability problems. 

Let P(x) be the statement “Student x knows calculus” and 

let Q(y) be the statement “Class y contains a student who 

knows calculus.” Express each of these as quantifications 

of P(x) and Q(y). 

a) Some students know calculus. 

b) Not every student knows calculus. 

c) Every class has a student in it who knows calculus. 

d) Every student in every class knows calculus. 

e) There is at least one class with no students who know 
calculus. 

Let P(m, n) be the statement “‘m divides n,” where the do- 

main for both variables consists of all positive integers. 

(By “‘m divides n” we mean that n = km for some integer 

k.) Determine the truth values of each of these statements. 


a) P(4,5) b) P(2,4) 
c) Vm Vn P(m,n) d) 4mVn P(m,n) 
e) dnVm P(m,n) f) Vn P(i,n) 


Find a domain for the quantifiers in axdy(x A yA 
Vz((z = x) V (z = y))) such that this statement is true. 


23. 


24, 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


Find a domain for the quantifiers in Axdy(x AYA 
Vz((z = x) V (z = y))) such that this statement is false. 


Use existential and universal quantifiers to express the 
statement “No one has more than three grandmothers” us- 
ing the propositional function G(x, y), which represents 
“x is the grandmother of y.” 


Use existential and universal quantifiers to express the 

statement “Everyone has exactly two biological parents” 

using the propositional function P(x, y), which repre- 

sents “‘x is the biological parent of y.” 

The quantifier 4, denotes “there exists exactly n,” so that 

dnx P(x) means there exist exactly n values in the do- 

main such that P(x) is true. Determine the true value of 

these statements where the domain consists of all real 

numbers. 

a) Jox(x* = —1) b) Jix(\x| = 0) 

©) Agx(x* = 2) d) A3x(x = |x|) 

Express each of these statements using existential and 

universal quantifiers and propositional logic where 4, is 

defined in Exercise 26. 

a) dox P(x) b) A,x P(x) 

c) dox P(x) d) 43x P(x) 

Let P(x, y) be a propositional function. Show that 

dx Vy P(x, y) > Vy Ax P(x, y) is a tautology. 

Let P(x) and Q(x) be propositional functions. Show 

thatax (P(x) > Q(x)) andVx P(x) > Ax Q(x) always 

have the same truth value. 

If Vy dx P(x, y) is true, does it necessarily follow that 

dx Vy P(x, y) is true? 

If Vx dy P(x, y) is true, does it necessarily follow that 

dx Vy P(x, y) is true? 

Find the negations of these statements. 

a) If it snows today, then I will go skiing tomorrow. 

b) Every person in this class understands mathematical 
induction. 

c) Some students in this class do not like discrete math- 
ematics. 

d) In every mathematics class there is some student who 
falls asleep during lectures. 


Computer Projects 


Write programs with the specified input and output. 


1. 


#3, 


Given the truth values of the propositions p and q, find the 
truth values of the conjunction, disjunction, exclusive or, 
conditional statement, and biconditional of these proposi- 
tions. 


. Given two bit strings of length n, find the bitwise AND, 


bitwise OR, and bitwise XOR of these strings. 


Give a compound proposition, determine whether it is sat- 
isfiable by checking its truth value for all positive assign- 
ments of truth values to its propositional variables. 


33. 


34. 


35. 


36. 


37. 


38. 
39. 


40. 


41. 


42. 


43. 


44, 


45. 


46. 


4. 


#5, 


*6. 
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Express this statement using quantifiers: “Every student 
in this class has taken some course in every department 
in the school of mathematical sciences.” 


Express this statement using quantifiers: “There is a build- 
ing on the campus of some college in the United States in 
which every room is painted white.” 


Express the statement “There is exactly one student in this 
class who has taken exactly one mathematics class at this 
school” using the uniqueness quantifier. Then express this 
statement using quantifiers, without using the uniqueness 
quantifier. 


Describe a rule of inference that can be used to prove that 
there are exactly two elements x and y in a domain such 
that P(x) and P(y) are true. Express this rule of inference 
as a statement in English. 

Use rules of inference to show that if the premises 
Vx (P(x) > Q(x)), VWx(Q(x) > R(x)), and —R(a), 
where a is in the domain, are true, then the conclusion 
=P (a) is true. 

Prove that if x? is irrational, then x is irrational. 

Prove that if x is irrational and x > 0, then ,/x is irra- 
tional. 

Prove that given a nonnegative integer n, there is a unique 
nonnegative integer m such that m2 <n < (m+ 1)?. 
Prove that there exists an integer m such that m? > 10109, 
Is your proof constructive or nonconstructive? 

Prove that there is a positive integer that can be written 
as the sum of squares of positive integers in two differ- 
ent ways. (Use a computer or calculator to speed up your 
work.) 

Disprove the statement that every positive integer is the 
sum of the cubes of eight nonnegative integers. 
Disprove the statement that every positive integer is the 
sum of at most two squares and a cube of nonnegative 
integers. 

Disprove the statement that every positive integer is the 
sum of 36 fifth powers of nonnegative integers. 
Assuming the truth of the theorem that states that ./n is 
irrational whenever n is a positive integer that is not a 
perfect square, prove that /2 + J/3 is irrational. 


Given the truth values of the propositions p and q in 
fuzzy logic, find the truth value of the disjunction and 
the conjunction of p and q (see Exercises 46 and 47 of 
Section 1.1). 


Given positive integers m and n, interactively play the game 
of Chomp. 


Given a portion of a checkerboard, look for tilings of this 
checkerboard with various types of polyominoes, including 
dominoes, the two types of triominoes, and larger polyomi- 
noes. 
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Computations and Explorations 


Use a computational program or programs you have written to do these exercises. 


1. Look for positive integers that are not the sum of the cubes 
of nine different positive integers. 

2. Look for positive integers greater than 79 that are not the 
sum of the fourth powers of 18 positive integers. 

3. Find as many positive integers as you can that can be writ- 
ten as the sum of cubes of positive integers, in two different 
ways, sharing this property with 1729. 


Writing Projects 


Respond to these with essays using outside sources. 


1. Discuss logical paradoxes, including the paradox of Epi- 
menides the Cretan, Jourdain’s card paradox, and the bar- 
ber paradox, and how they are resolved. 


2. Describe how fuzzy logic is being applied to practical ap- 
plications. Consult one or more of the recent books on 
fuzzy logic written for general audiences. 

3. Describe some of the practical problems that can be mod- 
eled as satisfiability problems. 

4. Describe some of the techniques that have been devised 
to help people solve Sudoku puzzles without the use of a 
computer. 

5. Describe the basic rules of WFF’N PROOF, The Game of 
Modern Logic, developed by Layman Allen. Give exam- 
ples of some of the games included in WFF’N PROOF. 

6. Read some of the writings of Lewis Carroll on symbolic 
logic. Describe in detail some of the models he used to 
represent logical arguments and the rules of inference he 
used in these arguments. 

7. Extend the discussion of Prolog given in Section 1.4, ex- 
plaining in more depth how Prolog employs resolution. 


*4, 


5. 


6. 


10. 


11. 


12. 


13. 


14. 


Try to find winning strategies for the game of Chomp for 
different initial configurations of cookies. 


Construct the 12 different pentominoes, where a pentomino 
is a polyomino consisting of five squares. 


Find all the rectangles of 60 squares that can be tiled using 
every one of the 12 different pentominoes. 


. Discuss some of the techniques used in computational 


logic, including Skolem’s rule. 


. “Automated theorem proving” is the task of using com- 


puters to mechanically prove theorems. Discuss the goals 
and applications of automated theorem proving and the 
progress made in developing automated theorem provers. 


Describe how DNA computing has been used to solve 
instances of the satisfiability problem. 


Look up some of the incorrect proofs of famous open 
questions and open questions that were solved since 1970 
and describe the type of error made in each proof. 


Discuss what is known about winning strategies in the 
game of Chomp. 


Describe various aspects of proof strategy discussed by 
George Polya in his writings on reasoning, including 
[Po62], [Po71], and [Po90]. 


Describe a few problems and results about tilings with 
polyominoes, as described in [Go94] and [Ma91], for ex- 
ample. 


2.1 Sets 
2.2. Set Operations 


2.3 Functions 


2.4 Sequences and 
Summations 


2.5 Cardinality of 
Sets 


2.6 Matrices 


IAL Sets 


a seers and eertaccp 


Mc of discrete mathematics is devoted to the study of discrete structures, used to repre- 
sent discrete objects. Many important discrete structures are built using sets, which are 
collections of objects. Among the 


These are some of the topics we will study 
in later chapters. 

The concept of a function is extremely important in discrete mathematics. A function assigns 
to each element of a first set exactly one element of a second set, where the two sets are not 
necessarily distinct.(Funetions play important roles throughout discrete mathematics. They are 
used to to study the size of sets, to count 
objects, and in a myriad of other ways. Useful structures such as sequences and strings are 
special types of functions. In this chapter, we will introduce the notion of a sequence, which 
represents ordered lists of elements. Furthermore, we will introduce some important types of 
sequences and we will show how to define the terms of a sequence using earlier terms. We will 
also address the problem of identifying a sequence from its first few terms. 

In our study of discrete mathematics, we will often add consecutive terms of a sequence of 
numbers. Because adding terms from a sequence, as well as other indexed sets of numbers, is 
such a common occurrence, a special notation has been developed for adding such terms. In this 
chapter, we will introduce the notation used to express summations. We will develop formulae 
for certain types of summations that appear throughout the study of discrete mathematics. For 
instance, we will encounter such summations in the analysis of the number of steps used by an 
algorithm to sort a list of numbers so that its terms are in increasing order. 

The relative sizes of infinite sets can be studied by introducing the notion of the size, or 
cardinality, of a set. We say that a set is countable when it is finite or has the same size as the 
set of positive integers. In this chapter we will establish the surprising result that the set of 
rational numbers is countable, while the set of real numbers is not. We will also show how the 
concepts we discuss can be used to show that there are functions that cannot be computed using 
a computer program in any programming language. 

Matrices are used in discrete mathematics to represent a variety of discrete structures. We 
will review the basic material about matrices and matrix arithmetic needed to represent relations 
and graphs. The matrix arithmetic we study will be used to solve a variety of problems involving 
these structures. 


Introduction 


In this section, we study the fundamental discrete structure on which all other discrete structures 
are built, namely, the set. 

For instance, all the students who are currently enrolled 
in your school make up a set. Likewise, all the students currently taking a course in discrete 
mathematics at any school make up a set. In addition, those students enrolled in your school 
who are taking a course in discrete mathematics form a set that can be obtained by taking the 
elements common to the first two collections. The language of sets is a means to study such 
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collections in an organized fashion. We now provide a definition of a set. This definition is an 
intuitive definition, which is not part of a formal theory of sets. 


(DEFINITIOND A ser is an unordered collection of objects, called elements or members of the set. A set is 


EXAMPLE 1 
EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


Extra p> 
Examples a 


Beware that mathe- 
maticians disagree 
whether 0 is a natural 
number. We consider it 
quite natural. 


said to contain its elements. We write a € A to denote that a is an element of the set A. The 
notation a ¢ A denotes that a is not an element of the set A. 


It is common for(§€t8 to be denoted using pperease letters./Lowercase letters are usually 
used to denote(€léments of sets. 


There are several ways to describe a set. One way is to list all the members of a set, when 
this is possible. We use a notation where all members of the set are listed between braces. For 
example, th This 
way of descr 


The set V of all vowels in the English alphabet can be written as V = {a, e, i, 0, u}. < 
The set O of odd positive integers less than 10 can be expressed by O = {1, 3, 5, 7, 9}. < 


Although sets are usually used to group together elements with common properties, there is 
nothing that prevents a set from having seemingly unrelated elements. For instance, {a, 2, Fred, 
New Jersey} is the set containing the four elements a, 2, Fred, and New Jersey. < 


Sometimes the roster method is used to describe a set without listing all its members. Some 
members of the set are listed, and then ellipses (...) are used when the general pattern of the 
elements is obvious. 


The set of positive integers less than 100 can be denoted by {1, 2,3, ..., 99}. < 


Another way to describe a set is to use set builder notation. We characterize all those 
elements in the set by stating the property or properties they must have to be members. For 
instance, the set O of all odd positive integers less than 10 can be written as 


O = {x | x is an odd positive integer less than 10}, 
or, specifying the universe as the set of positive integers, as 
O = {x €Z* | x is odd and x < 10}. 


We often use this type of notation to describe sets when it is impossible to list all the elements 
of the set. For instance, the set Qt of all positive rational numbers can be written as 


Qt={xeR|x= oe for some positive integers p and q}. 


N = {0, 1, 2,3, ...}, the set of natural numbers 
Z={...,—-2,—-1,0,1,2,...}, the set of integers 

Z* = {1,2,3,...}, the set of positive integers 
Q={p/q|p<éZ,q € Z,and g F O}, the set of rational numbers 
R, the set of real numbers 

R‘, the set of positive real numbers 

C, the set of complex numbers. 


EXAMPLE 5 


DEFINITION 2 


EXAMPLE 6 
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2.1 Sets 117 


(Note that some people do not consider 0 a natural number, so be careful to check how the term 
natural numbers is used when you read other books.) 

Recall the notation for intervals of real numbers. When a and Db are real numbers with 
a < b, we write 


la,b]={x|a<x <b} 
[a,b)={x|a<x <b} 
(a,b] = {x|a <x <b} 
(a,b) ={x|a <x <b} 


Note that [a, b] is called the closed interval from a to b and (a, b) is called the open interval 
from a to b. 
Sets can have other sets as members, as Example 5 illustrates. 


The set {N, Z, Q, R} is a set containing four elements, each of which 1s a set. The four elements 
of this set are N, the set of natural numbers; Z, the set of integers; Q, the set of rational numbers; 
and R, the set of real numbers. < 


Remark: Note that the concept of a datatype, or type, in computer science is built upon the 
concept of a set. In particular, a datatype or type is the name of a set, together with a set of 
operations that can be performed on objects from that set. For example, boolean is the name of 
the set {0, 1} together with operators on one or more elements of this set, such as AND, OR, 
and NOT. 


Because many mathematical statements assert that two differently specified collections of 
objects are really the same set, we need to understand what it means for two sets to be equal. 


Two sets are equal if and only if they have the same elements. Therefore, if A and B are sets, 
then A and B are equal if and only if Vx(x € A < x € B). We write A = B if A and B are 
equal sets. 


The sets {1, 3,5} and {3, 5, 1} are equal, because they have the same elements. Note that the 
order in which the elements of a set are listed does not matter. Note also that it does not matter 
if an element of a set is listed more than once, so {1, 3, 3, 3,5, 5,5, 5} is the same as the set 
{1, 3, 5} because they have the same elements. < 


GEORG CANTOR (1845-1918) Georg Cantor was born in St. Petersburg, Russia, where his father was a 
successful merchant. Cantor developed his interest in mathematics in his teens. He began his university studies 
in Zurich in 1862, but when his father died he left Zurich. He continued his university studies at the University 
of Berlin in 1863, where he studied under the eminent mathematicians Weierstrass, Kummer, and Kronecker. 
He received his doctor’s degree in 1867, after having written a dissertation on number theory. Cantor assumed 
a position at the University of Halle in 1869, where he continued working until his death. 

Cantor is considered the founder of set theory. His contributions in this area include the discovery that the 
set of real numbers is uncountable. He is also noted for his many important contributions to analysis. Cantor 
also was interested in philosophy and wrote papers relating his theory of sets with metaphysics. 


Cantor married in 1874 and had five children. His melancholy temperament was balanced by his wife’s happy disposition. 
Although he received a large inheritance from his father, he was poorly paid as a professor. To mitigate this, he tried to obtain a 
better-paying position at the University of Berlin. His appointment there was blocked by Kronecker, who did not agree with Cantor’s 
views on set theory. Cantor suffered from mental illness throughout the later years of his life. He died in 1918 from a heart attack. 
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{G} has one more 
element than 9. 


Links 


Assessment 


EXAMPLE 7 


THE EMPTY SET There is a special set that has no elements. This set is called the empty set, 
or null set, and is denoted by J. The empty set can also be denoted by { } (that is, we represent 
the empty set with a pair of braces that encloses all the elements in this set). Often, a set of 
elements with certain properties turns out to be the null set. For instance, the set of all positive 
integers that are greater than their squares is the null set. 

A set with one element is called a singleton set. A common error is to confuse the empty 
set J with the set {4}, which is a singleton set. The single element of the set {4} is the empty set 
itself! A useful analogy for remembering this difference is to think of folders in a computer file 
system. The empty set can be thought of as an empty folder and the set consisting of just the 
empty set can be thought of as a folder with exactly one folder inside, namely, the empty folder. 


NAIVE SET THEORY Note that the term object has been used in the definition of a set, 
Definition 1, without specifying what an object is. This description of a set as a collection 
of objects, based on the intuitive notion of an object, was first stated in 1895 by the German 
mathematician Georg Cantor. The theory that results from this intuitive definition of a set, and 
the use of the intuitive notion that for any property whatever, there is a set consisting of exactly 
the objects with this property, leads to paradoxes, or logical inconsistencies. This was shown 
by the English philosopher Bertrand Russell in 1902 (see Exercise 46 for a description of one of 
these paradoxes). These logical inconsistencies can be avoided by building set theory beginning 
with axioms. However, we will use Cantor’s original version of set theory, known as naive set 
theory, in this book because all sets considered in this book can be treated consistently using 
Cantor’s original theory. Students will find familiarity with naive set theory helpful if they go on 
to learn about axiomatic set theory. They will also find the development of axiomatic set theory 
much more abstract than the material in this text. We refer the interested reader to [Su72] to 
learn more about axiomatic set theory. 


Venn Diagrams 


Sets can be represented graphically using Venn diagrams, named after the English mathemati- 
cian John Venn, who introduced their use in 1881. In Venn diagrams the universal set U, which 
contains all the objects under consideration, is represented by a rectangle. (Note that the uni- 
versal set varies depending on which objects are of interest.) Inside this rectangle, circles or 
other geometrical figures are used to represent sets. Sometimes points are used to represent the 
particular elements of the set. Venn diagrams are often used to indicate the relationships between 
sets. We show how a Venn diagram can be used in Example 7. 


Draw a Venn diagram that represents V, the set of vowels in the English alphabet. 
Solution: We draw a rectangle to indicate the universal set U, which is the set of the 26 letters 


of the English alphabet. Inside this rectangle we draw a circle to represent V. Inside this circle 
we indicate the elements of V with points (see Figure 1). < 


FIGURE 1 Venn Diagram for the Set of Vowels. 


EXAMPLE 8 


EXAMPLE 9 
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Subsets 


It is common to encounter situations where the elements of one set are also the elements of 
a second set. We now introduce some terminology and notation to express such relationships 
between sets. 


The set A is a subset of B if and only if every element of A is also an element of B. We use 
the notation A C B to indicate that A is a subset of the set B. 


We see that A C B if and only if the quantification 
Vx(x Ee A>xe B) 


is true. Note that to show that A is not a subset of B we need only find one element x € A with 
x ¢ B. Such an x is a counterexample to the claim that x € A implies x € B. 
We have these useful rules for determining whether one set is a subset of another: 


Showing that A is a Subset of B To show that A C B, show that if x belongs to A then x 
also belongs to B. 


Showing that A is Not a Subset of B To show that A Z B, find a single x € A such that 
x ZB. 


The set of all odd positive integers less than 10 is a subset of the set of all positive integers less 
than 10, the set of rational numbers is a subset of the set of real numbers, the set of all computer 
science majors at your school is a subset of the set of all students at your school, and the set of 
all people in China is a subset of the set of all people in China (that is, it is a subset of itself). 
Each of these facts follows immediately by noting that an element that belongs to the first set 
in each pair of sets also belongs to the second set in that pair. < 


The set of integers with squares less than 100 is not a subset of the set of nonnegative integers 
because —1 is in the former set [as (—1)* < 100], but not the later set. The set of people who 
have taken discrete mathematics at your school is not a subset of the set of all computer science 
majors at your school if there is at least one student who has taken discrete mathematics who is 
not a computer science major. < 


BERTRAND RUSSELL (1872-1970) Bertrand Russell was born into a prominent English family active in 
the progressive movement and having a strong commitment to liberty. He became an orphan at an early age 
and was placed in the care of his father’s parents, who had him educated at home. He entered Trinity College, 
Cambridge, in 1890, where he excelled in mathematics and in moral science. He won a fellowship on the basis 
of his work on the foundations of geometry. In 1910 Trinity College appointed him to a lectureship in logic and 
the philosophy of mathematics. 

Russell fought for progressive causes throughout his life. He held strong pacifist views, and his protests 
against World War I led to dismissal from his position at Trinity College. He was imprisoned for 6 months in 
1918 because of an article he wrote that was branded as seditious. Russell fought for women’s suffrage in Great 


Britain. In 1961, at the age of 89, he was imprisoned for the second time for his protests advocating nuclear disarmament. 

Russell’s greatest work was in his development of principles that could be used as a foundation for all of mathematics. His 
most famous work is Principia Mathematica, written with Alfred North Whitehead, which attempts to deduce all of mathematics 
using a set of primitive axioms. He wrote many books on philosophy, physics, and his political ideas. Russell won the Nobel Prize 
for literature in 1950. 
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THEOREM 1 


FIGURE 2 Venn Diagram Showing that A Is a Subset of B. 


Theorem | shows that every nonempty set S is guaranteed to have at least two subsets, the 
empty set and the set S itself, that is,@ C Sand$ CS. 


For every set S,(@)@ CS and (@i)SCS. 


Proof: We will prove (7) and leave the proof of (ii) as an exercise. 


conditional statemen 


When we wish to emphasize that a set A is a subset of a set B but that A # B, we write 
A C B and say that A is a proper subset of B. For A C Bto be true, it must be the case that 
A C B and there must exist an element x of B that is not an element of A. That is, A is a proper 
subset of B if and only if 


Vx(x € A> xe B)AAX(XE BAX €A) 


is true. Venn diagrams can be used to illustrate that a set A is a subset of a set B. We draw the 
universal set U as arectangle. Within this rectangle we draw a circle for B. Because A is a subset 
of B, we draw the circle for A within the circle for B. This relationship is shown in Figure 2. 
A useful way to show that two sets have the same elements is to show that each set is a 
subset of the other. In other words, we can show that if A and B are sets with A C Band B C A, 
then A = B. That is, A = Bif and only if Vx(x € A > x € B) and Vx(x € B—> x € A) or 
equivalently if and only if Vx(x € A < x € B), which is what it means for the A and B to be 
equal. Because this method of showing two sets are equal is so useful, we highlight it here. 


JOHN VENN (1834-1923) John Venn was born into a London suburban family noted for its philanthropy. 
He attended London schools and got his mathematics degree from Caius College, Cambridge, in 1857. He was 
elected a fellow of this college and held his fellowship there until his death. He took holy orders in 1859 and, 
after a brief stint of religious work, returned to Cambridge, where he developed programs in the moral sciences. 
Besides his mathematical work, Venn had an interest in history and wrote extensively about his college and 
family. 

Venn’s book Symbolic Logic clarifies ideas originally presented by Boole. In this book, Venn presents a 
systematic development of a method that uses geometric figures, known now as Venn diagrams. Today these 
diagrams are primarily used to analyze logical arguments and to illustrate relationships between sets. In addition 


to his work on symbolic logic, Venn made contributions to probability theory described in his widely used textbook on that subject. 


DEFINITION 4 


EXAMPLE 10 
EXAMPLE 11 


EXAMPLE 12 


DEFINITION 5 
EXAMPLE 13 


Extra pe 
Examples “2 


DEFINITION 6 
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Showing Two Sets are Equal To show that two sets A and B are equal, show that A C B 
and B CA. 


Sets may have other sets as members. For instance, we have the sets 
A = {0, {a}, {b}, {a, b}} and B = {x | x is a subset of the set {a, b}}. 


Note that these two sets are equal, that is, A = B. Also note that {a} € A, buta ¢ A. 


The Size of a Set 


Sets are used extensively in counting problems, and for such applications we need to discuss 
the sizes of sets. 


Let S be a set. If there are exactly n distinct elements in S where n is a nonnegative integer, 
we say that S is a finite set and that n is the cardinality of S. The cardinality of S is denoted 
by |S]. 


Remark: The term cardinality comes from the common usage of the term cardinal number as 
the size of a finite set. 


Let A be the set of odd positive integers less than 10. Then |A| = 5. < 
Let S be the set of letters in the English alphabet. Then |S| = 26. < 
Because the null set has no elements, it follows that || = 0. < 


We will also be interested in sets that are not finite. 
A set is said to be infinite if it is not finite. 


The set of positive integers is infinite. < 


We will extend the notion of cardinality to infinite sets in Section 2.5, a challenging topic 
full of surprising results. 


Power Sets 


Many problems involve testing all combinations of elements of a set to see if they satisfy some 
property. To consider all such combinations of elements of a set S, we build a new set that has 
as its members all the subsets of S. 


Given a set S, the power set of S is the set of all subsets of the set S. The power set of S is 
denoted by P(S). 
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EXAMPLE 14 What is the power set of the set {0, 1, 2}? 


Extra * Solution: The power set P({0, 1, 2}) is the set of all subsets of {0, 1, 2}. Hence, 
Examples Qi 
P({O, 1, 2}) = {O, {O}, {1}, {2}, (0, 1}, {0, 2}, (1, 2}, (0, 1, 2}}. 


Note that the empty set and the set itself are members of this set of subsets. < 


EXAMPLE15 What is the power set of the empty set? What is the power set of the set {0}? 


Solution: The empty set has exactly one subset, namely, itself. Consequently, 
PO) = {9}. 
The set {G} has exactly two subsets, namely, @ and the set {@} itself. Therefore, 
P({D}) = (0, {03}. ” 


If a set has n elements, then its power set has 2” elements. We will demonstrate this fact in 
several ways in subsequent sections of the text. 


Cartesian Products 


The order of elements in a collection is often important. Because sets are unordered, a different 
structure is needed to represent ordered collections. This is provided by ordered n-tuples. 


DEFINITION 7 _ The ordered n-tuple (aj, a2, ..., dn) is the ordered collection that has ay as its first element, 
az as its second element, ..., and a, as its nth element. 


We say that two ordered n-tuples are equal if and only if each corresponding pair of their 
elements is equal. In other words, (a1, a2,..., dn) = (b1, b2,..., by) if and only if aj = bj, 
fori = 1,2,...,n. In particular, ordered 2-tuples are called ordered pairs. The ordered pairs 
(a, b) and (c, d) are equal if and only if a = c and b = d. Note that (a, b) and (b, a) are not 
equal unless a = b. 


RENE DESCARTES (1596-1650) René Descartes was born into a noble family near Tours, France, about 
200 miles southwest of Paris. He was the third child of his father’s first wife; she died several days after his 
birth. Because of René’s poor health, his father, a provincial judge, let his son’s formal lessons slide until, at 
the age of 8, René entered the Jesuit college at La Fléche. The rector of the school took a liking to him and 
permitted him to stay in bed until late in the morning because of his frail health. From then on, Descartes spent 
his mornings in bed; he considered these times his most productive hours for thinking. 

Descartes left school in 1612, moving to Paris, where he spent 2 years studying mathematics. He earned 
a law degree in 1616 from the University of Poitiers. At 18 Descartes became disgusted with studying and 
decided to see the world. He moved to Paris and became a successful gambler. However, he grew tired 
of bawdy living and moved to the suburb of Saint-Germain, where he devoted himself to mathematical study. When his gambling 
friends found him, he decided to leave France and undertake a military career. However, he never did any fighting. One day, while 
escaping the cold in an overheated room at a military encampment, he had several feverish dreams, which revealed his future career 
as a mathematician and philosopher. 

After ending his military career, he traveled throughout Europe. He then spent several years in Paris, where he studied mathemat- 
ics and philosophy and constructed optical instruments. Descartes decided to move to Holland, where he spent 20 years wandering 
around the country, accomplishing his most important work. During this time he wrote several books, including the Discours, which 
contains his contributions to analytic geometry, for which he is best known. He also made fundamental contributions to philosophy. 

In 1649 Descartes was invited by Queen Christina to visit her court in Sweden to tutor her in philosophy. Although he was 
reluctant to live in what he called “the land of bears amongst rocks and ice,” he finally accepted the invitation and moved to Sweden. 
Unfortunately, the winter of 1649-1650 was extremely bitter. Descartes caught pneumonia and died in mid-February. 
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Many of the discrete structures we will study in later chapters are based on the notion of the 
Cartesian product of sets (named after René Descartes). We first define the Cartesian product 
of two sets. 


(DEFINITIONS) = Let A and B be sets. The Cartesian product of A and B, denoted by A x B, is the set of all 
ordered pairs (a, b), where a € A andb é B. Hence, 


Ax B={(a,b)|aEeEANbe B}. 


EXAMPLE 16 Let A represent the set of all students at a university, and let B represent the set of all courses 
offered at the university. What is the Cartesian product A x B and how can it be used? 


Extra & Solution: The Cartesian product A x B consists of all the ordered pairs of the form (a, b), where 
Examples @ikd a is a student at the university and b is a course offered at the university. One way to use the set 
A x B is to represent all possible enrollments of students in courses at the university. < 


P00 SE 


Solution: The Cartesian product A x B is 


og BO A) < 


Note that the Cartesian products A x B and B x A are not equal, unless A = J or B = J 
(so that A x B = @) or A = B (see Exercises 31 and 38). This is illustrated in Example 18. 


EXAMPLE 18 — Show that the Cartesian product B x A is not equal to the Cartesian product A x B, where A 
and B are as in Example 17. 


Solution: The Cartesian product B x A is 
Bx A={@Q, 1), G, 2), ©, 1), ©, 2), (c, D, (c, 2)}- 
This is not equal to A x B, which was found in Example 17. < 


The Cartesian product of more than two sets can also be defined. 


DEFINITION 9 — The Cartesian product of the sets A,, A2,..., An, denoted by A; x Az x -:- x An, is the 
set of ordered n-tuples (a1, a2, ..., ,), where a; belongs to A; fori = 1,2,...,n. In other 
words, 


Aj x Aly aoe KA =n Wg ooo nia) || SE Ap ior? = 12%, scanidle 
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EXAMPLE 19 


EXAMPLE 20 


EXAMPLE 21 


What is the Cartesian product A x B x C, where A = {0, 1}, B = {1, 2}, and C = {0, 1, 2}? 


Solution: The Cartesian product A x B x C consists of all ordered triples (a, b, c), wherea € A, 
b € B,andc € C. Hence, 


Ax BxC= {(0, 1,0), (0, 1, 1), (0, 1,2), (0, 2, 0), (0, 2, 1), (0, 2, 2), 
(, 1,0), dd, 1, 1), d, 1, 2), C, 2, 0), C1, 2, 1), C, 2, 2)}. < 


Remark: Note that when A, B, and C are sets, (A x B) x C isnot the same as A x B x C (see 
Exercise 39). 


We use the notation A” to denote A x A, the Cartesian product of the set A with itself. 
Similarly, A? = A x A x A, A* = A x A x A x A, and so on. More generally, 


A” = {(a},a2,...,d,) | qa; € Afori=1,2,...,n}. 


Suppose that A ={1,2}. It follows that A? = {(1, 1), (1,2), (2,1), (2,2)} and A? = 
{, 1, 1), (1, 1, 2), d, 2, 1), C, 2, 2), 2, 1, 1), (2, 1, 2), (2, 2, 1), (2, 2, 2)}. 


A subset R of the Cartesian product A x B is called a relation from the set A to the set 
B. The elements of R are ordered pairs, where the first element belongs to A and the second 
to B. For example, R = {(a, 0), (a, 1), (a, 3), (b, 1), (B, 2), (c, 0), (c, 3)} is a relation from the 
set {a, b, c} to the set {0, 1, 2, 3}. A relation from a set A to itself is called a relation on A. 


What are the ordered pairs in the less than or equal to relation, which contains (a, b) if a < b, 
on the set {0, 1, 2, 3}? 


Solution: The ordered pair (a, b) belongs to R if and only if both a and b belong to {0, 1, 2, 3} 
and a < b. Consequently, the ordered pairs in R are (0,0), (0,1), (0,2), (0,3), (1,1), (1,2), C, “" 
(2,2), (2, 3), and (3, 3). 


We will study relations and their properties at length in Chapter 9. 


Sometimes we restrict the domain of a quantified statement explicitly by making use of a 
particular notation. For example, Vxe S(P(x)) denotes the universal quantification of P(x) 
over all elements in the set S. In other words, Vxe S(P(x)) is shorthand for Vx(x € S > P(x)). 
Similarly, xe S(P(x)) denotes the existential quantification of P(x) over all elements in S. 
That is, xe S(P(x)) is shorthand for ax(x € S A P(x)). 


Solution: The statement Vxe R(x? > 0) states that for every real number x, x? > 0. This state- 
ment can be expressed as “The square of every real number is nonnegative.” This is a true 
statement. 

The statement Jxe Z(x? = 1) states that there exists an integer x such that x? = 1. This 
statement can be expressed as “There is an integer whose square is 1.” This is also a true statement 
because x = | is such an integer (as is —1). < 
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We will now tie together concepts from set theory and from predicate logic. Given a predicate 
P, and a domain D, we define the truth set of P to be the set of elements x in D for which 
P(x) is true. The truth set of P(x) is denoted by {x € D | P(x)}. 


What are the truth sets of the predicates P(x), Q(x), and R(x), where the domain is the set of 
integers and P(x) is “|x| = 1,” Q(x) is “x2 — 2” and R(x) is “|x| = x.” 


Solution: The truth set of P, {x € Z | |x| = 1}, 1s the set of integers for which |x| = 1. Because 
|x| = 1 when x = | or x = —1, and for no other integers x, we see that the truth set of P is the 


set {—1, 1}. 


The truth set of O, {x € Z| x? = 2}, is the set of integers for which x? = 2. This is the 
empty set because there are no integers x for which x* = 2. 

The truth set of R, {x € Z| |x| = x}, is the set of integers for which |x| = x. Because 
|x| = x if and only if x > 0, it follows that the truth set of R is N, the set of nonnegative 


integers. 


< 


Note that Vx P(x) is true over the domain U if and only if the truth set of P is the set U. 
Likewise, 4x P(x) is true over the domain U if and only if the truth set of P is nonempty. 


Exercises 


1. List the members of these sets. 


a) {x | x is areal number such that x? = 1} 


5. Determine whether each of these pairs of sets are equal. 


@ 11,3, 3,3,5,5,5,5, 5}, (5,3, 1} 
b) {C3}, (1, 1 c) 9, {9} 


‘b) {x | x is a positive integer less than 12} 6. Suppose that A = {2, 4, 6}, B = {2, 6}, C = {4, 6}, and 
© {x | x is the square of an integer and x < 100} D= (4, 6, 8}. Determine which of these sets are subsets 
: ; 2 of which other of these sets. 
d) {x | x is an integer such that x“ = 2} : , ; 
: ' : wee 7. For each of the following sets, determine whether 2 is an 
. Use set builder notation to give a description of each of elamelit at Huai cel 
these sets. io RI xi . é ter than 1} 
a) {x € R|x is an integer greater than 
a) {0, 3, 6, 9, 12} b) {x € R| x is the square of an integer} 
Be eaele Deleeiat c) {2,{2}} d) {(2},({2}}} 
c) {m,n, 0, p} e) {{2},{2,{2}}} f) {{{2}}} 
. For each of these pairs of sets, determine whether the first 8. For each of the sets in Exercise 7, determine whether {2} 
is a subset of the second, the second is a subset of the first, is an element of that set. 
or neither is a subset of the other. 9. Determine whether each of these statements is true or 
(@)the set of airline flights from New York to New Delhi, false. 
the set of nonstop airline flights from New York to a) 0cg b) # € {0} 
New Delhi c) {0} cB d) 4c {0} 
b) the set of people who speak English, the set of people e) {0} € {0} f) {0} c {0} 
who speak Chinese g) {B} C {0} 
c) the set of flying squirrels, the set of living creatures 10. Determine whether these statements are true or false. 
that can fly a) % € {G} b) O € {9, {O}} 
. For each of these pairs of sets, determine whether the first c) {} € {9} d) {A} € {{}} 
is a subset of the second, the second is a subset of the first, e) {0} c {B, {By} f) {{A}} Cc {Y, {0} 
or neither is a subset of the other. g) {{D}} c {{B}, {(B}} 
a) the set of people who speak English, the set of people 11. Determine whether each of these statements is true or 
who speak English with an Australian accent false. 
b) the set of fruits, the set of citrus fruits a) xe {x} b) {x} C {x} c) {x} € {x} 
c) the set of students studying discrete mathematics, the d) {x} e€ {{x}} e) OC {x} f) Be {x} 
set of students studying data structures 12. Use a Venn diagram to illustrate the subset of odd integers 


in the set of all positive integers not exceeding 10. 
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13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


Use a Venn diagram to illustrate the set of all months of 
the year whose names do not contain the letter R in the 
set of all months of the year. 


Use a Venn diagram to illustrate the relationship A C B 
and BCC. 

Use a Venn diagram to illustrate the relationships A C B 
and BCC. 

Use a Venn diagram to illustrate the relationships A C B 
and ACC. 

Suppose that A, B, and C are sets such that A C B and 
B CC. Show that A CC. 


Find two sets A and B such that A € Band A C B. 
What is the cardinality of each of these sets? 

a) {a} b) {{a}} 

c) {a, {a}} d) {a, {a}, {a, {a}}} 
What is the cardinality of each of these sets? 

a) @ b) {9} 

c) {, {B}} d) {0, {H}, (9, {O}}} 


Find the power set of each of these sets, where a and b 
are distinct elements. 


a) {a} b) {a, b} c) {0, {H}} 
Can you conclude that A = B if A and B are two sets 
with the same power set? 


(23) How many elements does each of these sets have where 


24. 


25. 
26. 
27. 


28. 


29. 


30. 


31. 
32. 


a and Db are distinct elements? 

a) P({a, b, {a, b}}) 

b) P({O, a, {a}, {{a}}}) 

c) P(P@)) 

Determine whether each of these sets is the power set of 
a set, where a and BD are distinct elements. 

a) D b) {%, {a}} 

c) {A, {a}, {0, a}} d) {Y, {a}, {b}, {a, b}} 
Prove that P(A) C P(B) if and only if A C B. 

Show thatif AC Cand BC D,thenAx BCCxD 
Let A = {a, b,c, d} and B = {y, z}. Find 

a) Ax B. b) Bx A. 

What is the Cartesian product A x B, where A is the set 
of courses offered by the mathematics department at a 
university and B is the set of mathematics professors at 
this university? Give an example of how this Cartesian 
product can be used. 

What is the Cartesian product A x B x C, where A is 
the set of all airlines and B and C are both the set of all 
cities in the United States? Give an example of how this 
Cartesian product can be used. 

Suppose that A x B = J, where A and B are sets. What 
can you conclude? 


Let A be a set. Show that®@ x A=AxM=9. 
Let A = {a,b,c}, B = {x, y}, and C = {0, 1}. Find 
a) Ax BxC. b) Cx Bx A. 

ec) Cx AxB. d) Bx Bx B. 


33. 


34. 


35. 


36. 


37. 


38. 


39. 


40. 


Find A? if 

a) A= {0, 1, 3}. b) A= {1, 2,4, b}. 

Find A? if 

a) A= {a}. b) A = {0, a}. 

How many different elements does A x B have if A has 


m elements and B has n elements? 

How many different elements does A x B x C have if A 
has m elements, B has n elements, and C has p elements? 
How many different elements does A” have when A has 
m elements and n is a positive integer? 

Show that A x B # B x A, when A and B are nonempty, 
unless A = B. 

Explain why A x B x C and (A x B) x C are not the 
same. 

Explain why (A x B) x (C x D) and A x (B x C) x 
D are not the same. 


i) Translate each of these quantifications into English and 


42. 


43. 


44, 


* 45, 


* 46. 


* 47, 


determine its truth value. 

a) VxeER (x? 4-1) b) AxeZ (x? = 2) 

c) VxeEZ (x2 > 0) d) 4xER (x? = x) 

Translate each of these quantifications into English and 

determine its truth value. 

a) axeR (x? = —-1) b) axeZ (x +1> x) 

c) WxeEZ (x —1€Z) d) VxeZ (x? € Z) 

Find the truth set of each of these predicates where the 

domain is the set of integers. 

a) P(x): x2 <3 

ec) R(x): 2x +1=0 

Find the truth set of each of these predicates where the 

domain is the set of integers. 

a) P(x): >] 

c) R(x): x < x? 

The defining property of an ordered pair is that two or- 

dered pairs are equal if and only if their first elements 

are equal and their second elements are equal. Surpris- 

ingly, instead of taking the ordered pair as a primitive con- 

cept, we can construct ordered pairs using basic notions 

from set theory. Show that if we define the ordered pair 

(a, b) to be {{a}, {a, b}}, then (a, b) = (c, d) if and only 

ifa = candb = d. [Hint: First show that {{a}, {a, b}} = 

{{c}, {c, d}} if and only if a = c andb = d.] 

This exercise presents Russell’s paradox. Let S be the 

set that contains a set x if the set x does not belong to 

itself, so that S = {x | x ¢ x}. 

a) Show the assumption that S is a member of S leads to 
a contradiction. 

b) Show the assumption that S is nota member of S$ leads 
to a contradiction. 

By parts (a) and (b) it follows that the set § cannot be de- 

fined as it was. This paradox can be avoided by restricting 

the types of elements that sets can have. 


b) Q(x): x2 >x 


b) Q(x): x? =2 


Describe a procedure for listing all the subsets of a finite 
set. 
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Set Operations 


Links 


EXAMPLE 1 


EXAMPLE 2 


Introduction 


Two, or more, sets can be combined in many different ways. For instance, starting with the set 
of mathematics majors at your school and the set of computer science majors at your school, we 
can form the set of students who are mathematics majors or computer science majors, the set of 
students who are joint majors in mathematics and computer science, the set of all students not 
majoring in mathematics, and so on. 


Let A and B be sets. The union of the sets A and B, denoted by A U B, is the set that contains 
those elements that are either in A or in B, or in both. 


An element x belongs to the union of the sets A and B if and only if x belongs to A or x belongs 
to B. This tells us that 


AUB={x|xEAVxe Bh. 


The Venn diagram shown in Figure | represents the union of two sets A and B. The area 
that represents A U B is the shaded area within either the circle representing A or the circle 
representing B. 

We will give some examples of the union of sets. 


The union of the sets {1,3,5} and {1,2,3} is the set {1,2,3,5}; that is, 
{1, 3, 5} U {1, 2, 3} = {1, 2, 3, 5}. < 


The union of the set of all computer science majors at your school and the set of all mathe- 
matics majors at your school is the set of students at your school who are majoring either in 
mathematics or in computer science (or in both). 


Let A and B be sets. The intersection of the sets A and B, denoted by A/ B, is the set 
containing those elements in both A and B. 


An element x belongs to the intersection of the sets A and B if and only if x belongs to A and 
x belongs to B. This tells us that 


ANB={x|xEAAxe Bh. 


A U Bis shaded. AM Bis shaded. 


FIGURE 1 Venn Diagram of the FIGURE 2 Venn Diagram of the 
Union of A and B. Intersection of A and B. 
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EXAMPLE 3 


EXAMPLE 4 


EXAMPLE 5 


Be careful not to 
overcount! 


EXAMPLE 6 


EXAMPLE 7 


The Venn diagram shown in Figure 2 represents the intersection of two sets A and B. The shaded 
area that is within both the circles representing the sets A and B is the area that represents the 
intersection of A and B. 

We give some examples of the intersection of sets. 


The intersection of the sets {1,3,5} and {1,2,3} is the set {1,3}; that is, 
{1, 3,5} (1, 2, 3} = {1, 3}. < 


The intersection of the set of all computer science majors at your school and the set of all 
mathematics majors is the set of all students who are joint majors in mathematics and computer 
science. < 


Two sets are called disjoint if their intersection is the empty set. 


Let A = {1, 3,5, 7,9} and B = {2, 4, 6, 8, 10}. Because AN B = %, A and B are disjoint. <4 


We are often interested in finding the cardinality of a union of two finite sets A and B. Note 
that |A| + |B| counts each element that is in A but not in B or in B but not in A exactly once, 
and each element that is in both A and B exactly twice. Thus, if the number of elements that 
are in both A and B is subtracted from |A| + |B|, elements in A M B will be counted only once. 
Hence, 


|AU B| = |A|+ |B] —|AN BI. 


The generalization of this result to unions of an arbitrary number of sets is called the principle 
of inclusion—-exclusion. The principle of inclusion—exclusion is an important technique used in 
enumeration. We will discuss this principle and other counting techniques in detail in Chapters 6 
and 8. 

There are other important ways to combine sets. 


Let A and B be sets. The difference of A and B, denoted by A — B, is the set containing those 
elements that are in A but not in B. The difference of A and B is also called the complement 
of B with respect to A. 


Remark: The difference of sets A and B is sometimes denoted by A\B. 


An element x belongs to the difference of A and B if and only ifx € A and x ¢ B. This tells us 
that 


A-—B={x|xEAAx ¢ B}. 


The Venn diagram shown in(Figure 3 represents the(@ifference of the sets Aland! B)The shaded 


area inside the circle that represents A and outside the circle that represents B is the area that 
represents A — B. 
We give some examples of differences of sets. 


The difference of {1, 3,5} and {1, 2, 3} is the set {5}; that is, {1, 3, 5} — {1, 2, 3} = {5}. This 
is different from the difference of {1, 2, 3} and {1, 3, 5}, which is the set {2}. < 


The difference of the set of computer science majors at your school and the set of mathematics 
majors at your school is the set of all computer science majors at your school who are not also 
mathematics majors. 
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U U 
A-B is shaded. A is shaded. 
FIGURE 3 Venn Diagram for FIGURE 4 Venn Diagram for 
the Difference of A and B. the Complement of the Set A. 


Once the universal set U has been specified, the complement of a set can be defined. 


‘DEFINITION 5. Let U be the universal set. The complement of the set A, denoted by A, is the complement 


EXAMPLE 8 


EXAMPLE 9 


Set identities and 
propositional 
equivalences are just 
special cases of identities 
for Boolean algebra. 


of A with respect to U. Therefore, the complement of the set A is U — A. 


An element belongs to A if and only if x ¢ A. This tells us that 
A={xeU|x¢ A}. 


In Figure 4 the shaded area outside the circle representing A is the area representing A. 
We give some examples of the complement of a set. 


Let A = {a, e, i, o, u} (where the universal set is the set of letters of the English alphabet). Then 
A= {b,c,d, f, 8) kh, J, kl, M,N, DP, d,7, 8,1, V, Wi Xs Ys Zs 


Let A be the set of positive integers greater than 10 (with universal set the set of all positive 
integers). Then A = {1, 2,3, 4, 5, 6, 7, 8, 9, 10}. < 


It is left to the reader (Exercise 19) to show that we can express the difference of A and B 
as the intersection of A and the complement of B. That is, 


A-B=ANB. 


Set Identities 


Table 1 lists the most important set identities. We will prove several of these identities here, 
using three different methods. These methods are presented to illustrate that there are often many 
different approaches to the solution of a problem. The proofs of the remaining identities will 
be left as exercises. The reader should note the similarity between these set identities and the 
logical equivalences discussed in Section 1.3. (Compare Table 6 of Section 1.6 and Table 1.) In 
fact, the set identities given can be proved directly from the corresponding logical equivalences. 
Furthermore, both are special cases of identities that hold for Boolean algebra (discussed in 
Chapter 12). 

One way to show that two sets are equal is to show that each is a subset of the other. Recall 
that to show that one set is a subset of a second set, we can show that if an element belongs to 
the first set, then it must also belong to the second set. We generally use a direct proof to do this. 
We illustrate this type of proof by establishing the first of De Morgan’s laws. 
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EXAMPLE 10 


This identity says that 
the complement of the 
intersection of two sets 
is the union of their 
complements. 


Extra p> 
Examples 2 


TABLE 1 Set Identities. 


U(BUC)=(AUB)UC Associative laws 

N(BAC)=(ANB)AC 

U(BNC)=(AUB)N(AUC) Distributive laws 
(BUC) =(AN B)U(ANC) 


De Morgan’s laws 


AU(ANB)=A Absorption laws 
AN(AUB)=A 
A 


UA=U Complement laws 
ANA=9 


Prove that AN B= AUB. 


Solution: We will prove that the two sets AM B and A U B are equal by showing that each set 
is a subset of the other. 

(Fits8 we will show that AN B C AU B. We do this by showing that if x isin AN B, then it 
must also be in A U B. Now suppose that x € A B. By the definition of complement, x ¢ AN 
B. Using the definition of intersection, we see that the proposition =((x € A) A (x € B)) is true. 

By applying De Morgan's law for propositions) we see that =(x € A) or +(x € B). Using 
the definition of negation of propositions, we have x ¢ A or x ¢ B. Using the definition of 
the complement of a set, we see that this implies that x € A or x € B. Consequently, by the 
definition of union, we see that x € A U B. We have now shown that AN B C AUB. 

(NEX® we will show that AU B C AN B. We do this by showing that if x is in A U B, then 
it must also be in AN B. Now suppose that x € A U B. By the definition of union, we know that 
x € Aorx € B. Using the definition of complement, we see that x ¢ A orx ¢ B. Consequently, 
the proposition =(x € A) V 7(x € B) is true. 

By De)Morgan’s law for propositions, we conclude that —((x € A) A (x € B)) is true. 
By the definition of intersection, it follows that —(x € AM B). We now use the definition of 
complement to conclude that x € AM B. This shows that AU B C ANB. 

Because we have shown that each set is a subset of the other, the two sets are equal, and the 
identity is proved. < 


We can more succinctly express the reasoning used in Example 10 using set builder notation, 
as Example 11 illustrates. 


EXAMPLE 11 


EXAMPLE 12 


EXAMPLE 13 
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Use set builder notation and logical equivalences to establish the first De Morgan law AM B = 
AUB. 


Solution: We can prove this identity with the following steps. 


ANB={x|x¢ANB} by definition of complement 

{x | -=(x € (AN B))} by definition of does not belong symbol 
={x|7~weAAxe B)} by definition of intersection 
= {x | ->( € A) V-7(x € B)} | by the first De Morgan law for logical equivalences 
={x|x ¢éAvVx ¢€ B} by definition of does not belong symbol 
=(x|xe Avex € B) by definition of complement 

{x |x € AUB} by definition of union 
=AUB by meaning of set builder notation 


Note that besides the definitions of complement, union, set membership, and set builder 
notation, this proof uses the second De Morgan law for logical equivalences. 


Proving a set identity involving more than two sets by showing each side of the identity is 
a subset of the other often requires that we keep track of different cases, as illustrated by the 
proof in Example 12 of one of the distributive laws for sets. 


Prove the second distributive law from Table 1, which states that AN (BUC) =(ANMB)U 
(A 1 C) for all sets A, B, and C. 


Solution: We will prove this identity by showing that each side is a subset of the other side. 

Suppose that x € AN (BUC). Then x € A and x € BUC. By the definition of union, it 
follows that x € A, and x € B or x € C (or both). In other words, we know that the compound 
proposition (x € A) A ((x € B) V (x € C)) is true. By the distributive law for conjunction over 
disjunction, it follows that ((x € A) A (x € B)) V (jx € A) A (x € C)). We conclude that either 
x € Aandx € B,orx € Aandx € C. By the definition of intersection, it follows thatx ¢ AN B 
or x € ANC. Using the definition of union, we conclude that x € (AN B)U(ANC). We 
conclude that AN (BUC) C (AN B)U(ANC). 

Now suppose that x € (AM B)U (ANC). Then, by the definition of union, x € AM B or 
x € AMC. By the definition of intersection, it follows that x €¢ A and x € B or that x € A and 
x € C. From this we see that x € A, and x € B or x € C. Consequently, by the definition of 
union we see that x € A andx € BUC. Furthermore, by the definition of intersection, it follows 
that x € AN (BUC). We conclude that (ANB) U(ANC) C AN(BUC). This completes 
the proof of the identity. < 


Set identities can also be proved using membership tables. We consider each combination 
of sets that an element can belong to and verify that elements in the same combinations of sets 
belong to both the sets in the identity. To indicate that an element is in a set, a | is used; to 
indicate that an element is not in a set, a 0 is used. (The reader should note the similarity between 
membership tables and truth tables.) 


Use a membership table to show that AN (BUC) = (AN B)U(ANC). 
Solution: The membership table for these combinations of sets is shown in Table 2. This table 


has eight rows. Because the columns for AN (B U C) and (AN B) U (ANC) are the same, the 
identity is valid. < 


Additional set identities can be established using those that we have already proved. Consider 
Example 14. 
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TABLE 2 A Membership Table for the Distributive Property. 
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1 
1 
1 
1 
0 
0 
0 
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EXAMPLE 14 Let A, B, and C be sets. Show that 


AU(BNC) = (CUB)NA. 


Solution: We have 


AU(BNC)=AN (BNC) by the first De Morgan law 
= AN(BUC) by the second De Morgan law 
=(BUC)NA by the commutative law for intersections 


= (CUB)NMA_ by the commutative law for unions. 


Generalized Unions and Intersections 


Because unions and intersections of sets satisfy associative laws, the sets AU BUC and 
AM BOC are well defined; that is, the meaning of this notation is unambiguous when A, 
B, and C are sets. That is, we do not have to use parentheses to indicate which operation 
comes first because AU (BUC) = (AU B)UC and AN (BNC) = (AN B) NC. Note that 
AU BUC contains those elements that are in at least one of the sets A, B, and C, and that 
A™MBQC contains those elements that are in all of A, B, and C. These combinations of the 
three sets, A, B, and C, are shown in Figure 5. 


. & 
XS IS 


(a) A UB U Cis shaded. (b) A NB NMC is shaded. 


FIGURE 5 The Union and Intersection of A, B, and C. 


EXAMPLE 15 


EXAMPLE 16 
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Let A = {0, 2,4, 6, 8}, B = {0,1,2,3,4}, and C = {0,3, 6,9}. What are AU BUC and 
ANBNC? 


Solution: The set AU BUC contains those elements in at least one of A, B, and C. Hence, 
AUBUC = (0, 1, 2, 3, 4, 6, 8, 9}. 
The set AM BMC contains those elements in all three of A, B, and C. Thus, 


ANBNC = {0}. ST 


We can also consider unions and intersections of an arbitrary number of sets. We introduce 
these definitions. 


The union of a collection of sets is the set that contains those elements that are members of 
at least one set in the collection. 


We use the notation 
n 
AU APA. =| JA 
i=l 


to denote the union of the sets Aj, A2,..., An. 


The intersection of a collection of sets is the set that contains those elements that are members 
of all the sets in the collection. 


We use the notation 
n 
ALN ANN An =[ ) Ai 
i=l 


to denote the intersection of the sets A;, A2,..., An. We illustrate generalized unions and 
intersections with Example 16. 


Fori = 1,2,..., let Aj = {i,i +1,i+2,...}. Then, 
n n 
Ap | aa hl Osc S12, Bach; 
i=l] i=l 


and 


n n 
(\Ai=( it 1it2,...) =(nn4+1n4+2,...}= An. < 
i=l i=1 
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EXAMPLE 17 


EXAMPLE 18 


We can extend the notation we have introduced for unions and intersections to other families of 
sets. In particular, we use the notation 


CO 
AU AQU++-UAnU+++ =| J Ai 
i=l 
to denote the union of the sets A,, A2,..., An, ... . Similarly, the intersection of these sets is 
denoted by 
CO 
ALN AQN++-N An N+ =f) Ai. 


i=l 


More generally, when / is a set, the notations (};.,; Aj and ;<, Ai are used to denote 
the intersection and union of the sets A; fori € J, respectively. Note that we have ();.; Ai = 
{x | Vi e I(x € Aj)} and U),.; Ai = {x | di e I (x € Aj}. 


ie] 


Suppose that A; = {1,2,3,...,i} fori = 1,2,3,.... Then, 


[o) CO 
(JAH IO 2 e tht? sda ee 


i=1 i=1 


and 


(Qe Wee ee 


i=! i=1 


To see that the union of these sets is the set of positive integers, note that every positive 
integer n is in at least one of the sets, because it belongs to A, = {1, 2,...,m}, and every element 
of the sets in the union is a positive integer. To see that the intersection of these sets is the set 
{1}, note that the only element that belongs to all the sets Aj, Az, ... is 1. To see this note that 
A; ={l}and1 eA; fori =1,2,.... < 


Computer Representation of Sets 


There are various ways to represent sets using a computer. One method is to store the elements 
of the set in an unordered fashion. However, if this is done, the operations of computing the 
union, intersection, or difference of two sets would be time-consuming, because each of these 
operations would require a large amount of searching for elements. We will present a method 
for storing elements using an arbitrary ordering of the elements of the universal set. This method 
of representing sets makes computing combinations of sets easy. 

Assume that the universal set U is finite (and of reasonable size so that the number of 
elements of U is not larger than the memory size of the computer being used). First, specify an 
arbitrary ordering of the elements of U, for instance aj, a2, ..., dn. Represent a subset A of U 
with the bit string of length n, where the ith bit in this string is | if a; belongs to A and is 0 if 
a; does not belong to A. Example 18 illustrates this technique. 


Let U = {1,2,3,4,5, 6, 7, 8,9, 10}, and the ordering of elements of U has the elements in 
increasing order; that is, a; = i. What bit strings represent the subset of all odd integers in U, 
the subset of all even integers in U, and the subset of integers not exceeding 5 in U? 


EXAMPLE 19 


EXAMPLE 20 
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Solution: The bit string that represents the set of odd integers in U, namely, {1, 3, 5, 7, 9}, has 
a one bit in the first, third, fifth, seventh, and ninth positions, and a zero elsewhere. It is 


10 1010 1010. 


(We have split this bit string of length ten into blocks of length four for easy reading.) Similarly, 
we represent the subset of all even integers in U, namely, {2, 4, 6, 8, 10}, by the string 


01 0101 0101. 


The set of all integers in U that do not exceed 5, namely, {1, 2,3, 4, 5}, is represented by the 
string 


11 1110 0000. < 


Using bit strings to represent sets, it is easy to find complements of sets and unions, inter- 
sections, and differences of sets. To find the bit string for the complement of a set from the bit 
string for that set, we simply change each | to a 0 and each 0 to 1, because x € A if and only if 
x ¢ A. Note that this operation corresponds to taking the negation of each bit when we associate 
a bit with a truth value—with | representing true and 0 representing false. 


We have seen that the bit string for the set {1,3,5,7,9} (with universal set {1, 2, 3, 4, 
5, 6, 7, 8, 9, 10}) is 


10 1010 1010. 
What is the bit string for the complement of this set? 


Solution: The bit string for the complement of this set is obtained by replacing Os with 1s and 
vice versa. This yields the string 


01 0101 0101, 
which corresponds to the set {2, 4, 6, 8, 10}. < 


To obtain the bit string for the union and intersection of two sets we perform bitwise Boolean 
operations on the bit strings representing the two sets. The bit in the 7th position of the bit string 
of the union is | if either of the bits in the 7th position in the two strings is 1 (or both are 1), and 
is 0 when both bits are 0. Hence, the bit string for the union is the bitwise OR of the bit strings 
for the two sets. The bit in the ith position of the bit string of the intersection is 1 when the bits 
in the corresponding position in the two strings are both 1, and is 0 when either of the two bits 
is O (or both are). Hence, the bit string for the intersection is the bitwise AND of the bit strings 
for the two sets. 


The bit strings for the sets {1, 2, 3, 4, 5} and {1, 3, 5, 7, 9} are 11 1110 0000 and 10 1010 1010, 
respectively. Use bit strings to find the union and intersection of these sets. 


Solution: The bit string for the union of these sets is 
11 1110 0000 v 10 1010 1010 = 11 1110 1010, 


which corresponds to the set {1, 2, 3, 4,5, 7, 9}. The bit string for the intersection of these sets 
is 


11 1110 0000 A 10 1010 1010 = 10 1010 0000, 


which corresponds to the set {1, 3, 5}. < 
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Exercises 


Get A be the set of students who live within one mile 


of school and let B be the set of students who walk to 
classes. Describe the students in each of these sets. 


a) ANB b) AUB 
c) A-B d) B-A 
. Suppose that A is the set of sophomores at your school 


and B is the set of students in discrete mathematics at 
your school. Express each of these sets in terms of A and 
B. 


a) the set of sophomores taking discrete mathematics in 
your school 

b) the set of sophomores at your school who are not tak- 
ing discrete mathematics 

c) the set of students at your school who either are sopho- 
mores or are taking discrete mathematics 

d) the set of students at your school who either are not 
sophomores or are not taking discrete mathematics 


@) Let A = {1, 2, 3, 4, 5} and B = (0, 3, 6}. Find 


a) AUB. b) ANB. 

c) A-B. d) B-A. 

. Let A = {a,b,c,d,e} and B= {a,b,c,d,e, f, g,h}. 
Find 

a) AUB. b) ANB. 

c) A-B. d) B-A. 


In Exercises 5—10 assume that A is a subset of some underly- 
ing universal set U. 


5. 


14. 


15. 


Prove the complementation law in Table 1 by showing 
that A= A. 


. Prove the identity laws in Table 1 by showing that 
a) AUG=A. b) ANU=A. 

. Prove the domination laws in Table 1 by showing that 
a) AUU=U. b) ANG=4%. 

. Prove the idempotent laws in Table 1 by showing that 
a) AUA=A. b) ANA=A. 

. Prove the complement laws in Table 1 by showing that 
a) AUA=U. b) ANA=9. 

. Show that 
a) A-Y=A. b) J-A=9. 


. Let A and B be sets. Prove the commutative laws from 


Table 1 by showing that 


a) AUB=BUA. 
b) ANB=BNA. 


. Prove the first absorption law from Table 1 by showing 


that if A and B are sets, then AU(AN B) =A. 


. Prove the second absorption law from Table | by showing 


that if A and B are sets, then AN (AU B) = A. 

Find the sets A and B if A— B = {1,5,7,8}, B-—A= 
{2, 10}, and AN B = {3, 6, 9}. 

Prove the second De Morgan law in Table 1 by showing 
that if A and B are sets, then AUB=ANB 

a) by showing each side is a subset of the other side. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


b) using a membership table. 


Let A and B be sets. Show that 
a) (AN B)CA. b) AC (AUB). 
c) A-BCA. d) AN(B—A)=8%. 


e) AU(B—A)=AUB. 

Show that if A, B, and C are sets, then AN BNC = 
AUBUC 

a) by showing each side is a subset of the other side. 

b) using a membership table. 

Let A, B, and C be sets. Show that 

a) (AUB) C(AUBUC). 

b) (AN BNC) C (ANB). 

ce) (A—B)-CCA-C. 

d) (A-OC)N(C-B)=%9. 

e) (B—A)U(C—A)=(BUC)-A. 

Show that if A and B are sets, then 

a) A—B=ANB. 

b) (ANB) U(ANB) =A. 

Show that if A and B are sets with A C B, then 

a) AUB=B. 

b) ANB=A. 

Prove the first associative law from Table | by show- 
ing that if A, B, and C are sets, then AU(BUC) = 
(AU B)UC. 
Prove the second associative law from Table | by show- 
ing that if A, B, and C are sets, then AN (BNC) = 
(AN B)NC. 
Prove the first distributive law from Table 1 by show- 
ing that if A, B, and C are sets, then AU(BNC) = 
(AUB)N(AUC). 

Let A, B, and C be sets. Show that (A — B) -—C= 
(A—C)—(B-C). 

Let A = {0, 2,4, 6, 8, 10}, B = {0, 1,2, 3, 4,5, 6}, and 
C = {4,5, 6, 7, 8, 9, 10}. Find 

a) ANBNC. b) AUBUC. 

c) (AUB)NC. d) (AN B)UC. 

Draw the Venn diagrams for each of these combinations 
of the sets A, B, and C. 

a) AN(BUC) b) ANBNC 

c) (A— B)U(A-C)U(B-C) 

Draw the Venn diagrams for each of these combinations 
of the sets A, B, and C. 
a) AN(B-C) 

c) (AN B)U(ANC) 
Draw the Venn diagrams for each of these combinations 
of the sets A, B, C, and D. 

a) (AN B)U(CND) b) AUBUCUD 

c) A—(BNCND) 


b) (AN B)U(ANC) 


What can you say about the sets A and B if we know that 
a) AUB= A? b) ANB=A? 
c) A—~B=A? d) ANB=BNA? 


e) A—-B=B-A? 


30. 


31. 


Can you conclude that A = Bif A, B, and C are sets such 
that 


a) AUC=BUC? b) ANC=BNC? 
ec) AUC=BUCandANC=BNC? 


Let A and B be subsets of a universal set U. Show that 
A C Bifandonly if BC A. 


The symmetric difference of A and B, denoted by A @ B, is 
the set containing those elements in either A or B, but not in 
both A and B. 


32. 


33. 


34. 


35. 
36. 
37. 


38. 


39. 


* 40. 


*41, 


42. 


43. 


44. 


45. 


* 46. 


47. 


48. 


Find the symmetric difference of {1, 3, 5} and {1, 2, 3}. 


Find the symmetric difference of the set of computer sci- 
ence majors at a school and the set of mathematics majors 
at this school. 


Draw a Venn diagram for the symmetric difference of the 
sets A and B. 

Show that A@ B = (AUB) — (ANB). 

Show that A @ B = (A — B) U(B-— A). 

Show that if A is a subset of a universal set U, then 

a) A@A=¥%. b) A@Y=A. 

c) ABUZ=A. d) A@A=U. 

Show that if A and B are sets, then 

a) A®B=BOA. b) (A®B)G@B=A. 
What can you say about the sets A and B if A@ B = A? 


Determine whether the symmetric difference is associa- 
tive; that is, if A, B, and C are sets, does it follow that 
A®(BEC)=(AGB)GC? 

Suppose that A, B, and C are sets such that A®C = 
B@®C. Must it be the case that A = B? 


If A, B, C, and D are sets, does it follow that (A 6 B) @ 
(C@D)=(A®C)@(BOD)? 
If A, B, C, and D are sets, does it follow that (A 6 B) @ 
(C@®D)=(AGD)O(BEC)? 


Show that if A and B are finite sets, then A U B isa finite 
set. 


Show that if A is an infinite set, then whenever B is a set, 
A U Bis also an infinite set. 


Show that if A, B, and C are sets, then 


|AU BUC|=|A|+ |B] +|C| —|AN BI 
—|ANC|—|BNC|+|ANBNC|. 


(This is a special case of the inclusion—exclusion princi- 
ple, which will be studied in Chapter 8.) 


Let A; = {1,2,3,...,i} fori = 1,2,3,.... Find 


a) Ua. b) are 
i=1 i=l 


Let Aj = {...,-2,—-1,0,1,..., i}. Find 


a) (Ai. b) (Ai. 
i=1 i=1 


49. 


50. 


51. 


52. 


53. 


54. 


55. 


56. 


57. 


58. 
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Let A; be the set of all nonempty bit strings (that is, bit 
strings of length at least one) of length not exceeding i. 


Find 
n 
b) (Ai. 
i=1 


n 
a) J Ai. 
i=1 


Find P<, A; and (72, A; if for every positive integer i, 
a) Aj = {i,i+1,i4+2,...}. 


b) A; = {0, 7}. 
c) A; = (0,7), that is, the set of real numbers x with 
0O<x <i. 


d) A; = (i, &), that is, the set of real numbers x with 
x >i. 
Find P<, Ai and (72, A; if for every positive integer i, 


a) Aj ={-i,-i+1,...,—-1,0,1,...,i-1, i}. 

b) A; = {-i, i}. 

c) A; =[-i, i], that is, the set of real numbers x with 
-i<x <i. 


d) A; = [i, ©), that is, the set of real numbers x with 
x >t. 

Suppose that the universal set is U = {1,2,3,4, 

5, 6,7, 8,9, 10}. Express each of these sets with bit 

strings where the ith bit in the string is 1 if i is in the 

set and 0 otherwise. 


a) {3,4,5} 

b) {1, 3, 6, 10} 

c) {2,3,4, 7, 8, 9} 

Using the same universal set as in the last problem, find 
the set specified by each of these bit strings. 

a) 1111001111 

b) 010111 1000 

c) 100000 0001 


What subsets of a finite universal set do these bit strings 
represent? 

a) the string with all zeros 

b) the string with all ones 

What is the bit string corresponding to the difference of 
two sets? 


What is the bit string corresponding to the symmetric dif- 
ference of two sets? 


Show how bitwise operations on bit strings can be 
used to find these combinations of A = {a, b,c, d, e}, 
B = {b, c,d, g, p,t, v}, C = {c,e,i,0,u,x, y, z}, and 
D = {d,e,h,i,n,o,t,u,x, y}. 

a) AUB b) ANB 

c) (AUD)N(BUC) d) AUBUCUD 

How can the union and intersection of n sets that all are 
subsets of the universal set U be found using bit strings? 


The successor of the set A is the set A U {A}. 


59. 


Find the successors of the following sets. 
a) {1, 2, 3} b) @ 
c) {9} d) {0, {9}} 
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60. How many elements does the successor of a set with n 
elements have? 


Sometimes the number of times that an element occurs in an 
unordered collection matters. Multisets are unordered collec- 
tions of elements where an element can occur as a member 
more than once. The notation {m  - a), m2-a2,...,m,- ar} 
denotes the multiset with element a; occurring m , times, el- 
ement a2 occurring m2 times, and so on. The numbers m;, 
i=1,2,...,r are called the multiplicities of the elements 

GH V2) PS 
Let P and Q be multisets. The union of the multisets P 

and Q is the multiset where the multiplicity of an element is 

the maximum of its multiplicities in P and Q. The intersec- 
tion of P and Q is the multiset where the multiplicity of an 
element is the minimum of its multiplicities in P and Q. The 
difference of P and Q is the multiset where the multiplicity 
of an element is the multiplicity of the element in P less its 
multiplicity in Q unless this difference is negative, in which 
case the multiplicity is 0. The sum of P and Q is the multiset 
where the multiplicity of an element is the sum of multiplic- 
ities in P and Q. The union, intersection, and difference of 

P and Q are denoted by P U Q, PM Q, and P — Q, respec- 

tively (where these operations should not be confused with 

the analogous operations for sets). The sum of P and Q is 

denoted by P+ Q. 

61. Let A and B be the multisets {3-a,2-b,1-c} and 
{2-a,3-b,4- d}, respectively. Find 
a) AUB. b) ANB. 

d) B-A. e) A+B. 

62. Suppose that A is the multiset that has as its elements 
the types of computer equipment needed by one depart- 
ment of a university and the multiplicities are the number 
of pieces of each type needed, and B is the analogous 
multiset for a second department of the university. For 
instance, A could be the multiset {107 - personal comput- 
ers, 44 - routers, 6 - servers} and B could be the multiset 
{14 - personal computers, 6 - routers, 2 - mainframes}. 


c) A-B. 


a) What combination of A and B represents the equip- 
ment the university should buy assuming both depart- 
ments use the same equipment? 


Functions 


b) What combination of A and B represents the equip- 
ment that will be used by both departments if both 
departments use the same equipment? 

c) What combination of A and B represents the equip- 
ment that the second department uses, but the first de- 
partment does not, if both departments use the same 
equipment? 

d) What combination of A and B represents the equip- 
ment that the university should purchase if the depart- 
ments do not share equipment? 


Fuzzy sets are used in artificial intelligence. Each element 
in the universal set U has a degree of membership, which 
is a real number between 0 and | (including 0 and 1), in a 
fuzzy set S. The fuzzy set S is denoted by listing the elements 
with their degrees of membership (elements with 0 degree of 
membership are not listed). For instance, we write {0.6 Alice, 
0.9 Brian, 0.4 Fred, 0.1 Oscar, 0.5 Rita} for the set F' (of fa- 
mous people) to indicate that Alice has a 0.6 degree of mem- 
bership in F’,, Brian has a 0.9 degree of membership in F, Fred 
has a 0.4 degree of membership in F’, Oscar has a 0.1 degree 
of membership in F,, and Rita has a 0.5 degree of membership 
in F (so that Brian is the most famous and Oscar is the least 
famous of these people). Also suppose that R is the set of rich 
people with R = {0.4 Alice, 0.8 Brian, 0.2 Fred, 0.9 Oscar, 
0.7 Rita}. 


63. The complement of a fuzzy set S is the set S, with the 
degree of the membership of an element in S equal to 
1 minus the degree of membership of this element in S. 
Find F (the fuzzy set of people who are not famous) and 
R (the fuzzy set of people who are not rich). 


64. The union of two fuzzy sets S and T is the fuzzy set 
SU T, where the degree of membership of an element in 
SU T is the maximum of the degrees of membership of 
this element in S and in 7. Find the fuzzy set F U R of 
rich or famous people. 

65. The intersection of two fuzzy sets S and T is the fuzzy 
set SM T, where the degree of membership of an element 
in SM T is the minimum of the degrees of membership 
of this element in S and in 7. Find the fuzzy set FO R 
of rich and famous people. 


Introduction 


In many instances we assign to each element of a set a particular element of a second set (which 
may be the same as the first). For example, suppose that each student in a discrete mathematics 
class is assigned a letter grade from the set {A, B, C, D, F}. And suppose that the grades are A 
for Adams, C for Chou, B for Goodfriend, A for Rodriguez, and F for Stevens. This assignment 


of grades is illustrated in Figure 1. 


This assignment is an example of a function. The concept of a function is extremely impor- 
tant in mathematics and computer science. For example, in discrete mathematics functions are 
used in the definition of such discrete structures as sequences and strings. Functions are also 
used to represent how long it takes a computer to solve problems of a given size. Many computer 
programs and subroutines are designed to calculate values of functions. Recursive functions, 


DEFINITION 1 


Assessment 


DEFINITION 2 


2.3 Functions 139 


Adams e =eA 
Chou e eB 
Goodfriend e ec 
Rodriguez e@ eD 
Stevens e eel 


FIGURE 1 Assignment of Grades in a Discrete Mathematics Class. 


which are functions defined in terms of themselves, are used throughout computer science; they 
will be studied in Chapter 5. This section reviews the basic concepts involving functions needed 
in discrete mathematics. 


Let A and B be nonempty sets. A function f from A to B is an assignment of exactly one 
element of B to each element of A. We write f(a) = b if b is the unique element of B 
assigned by the function f to the element a of A. If f is a function from A to B, we write 
f: Av B. 


Remark: Functions are sometimes also called mappings or transformations. 


Functions are specified in many different ways. Sometimes we explicitly state the assign- 
ments, as in Figure 1. Often we give a formula, such as f(x) = x + 1, to define a function. 
Other times we use a computer program to specify a function. 

A function f : A > B can also be defined in terms of a relation from A to B. Recall from 
Section 2.1 that a relation from A to B is just a subset of A x B.A relation from A to B that 
contains one, and only one, ordered pair (a, b) for every element a € A, defines a function f 
from A to B. This function is defined by the assignment f(a) = b, where (a, b) is the unique 
ordered pair in the relation that has a as its first element. 


If f is a function from A to B, we say that A is the domain of f and B is the codomain of f. 
If f(a) = b, we say that b is the image of a and a is a preimage of b. The range, or image, 
of f is the set of all images of elements of A. Also, if f is a function from A to B, we say 
that fmaps A to B. 


Figure 2 represents a function f from A to B. 

When we define a function we specify its domain, its codomain, and the mapping of elements 
of the domain to elements in the codomain. Two functions are equal when they have the same 
domain, have the same codomain, and map each element of their common domain to the same 
element in their common codomain. Note that if we change either the domain or the codomain 


he ee 


f 
FIGURE 2 The Function f Maps A to B. 
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EXAMPLE 4 


EXAMPLE 5 


of a function, then we obtain a different function. If we change the mapping of elements, then 
we also obtain a different function. 

Examples 1-5 provide examples of functions. In each case, we describe the domain, the 
codomain, the range, and the assignment of values to elements of the domain. 


What are the domain, codomain, and range of the function that assigns grades to students 
described in the first paragraph of the introduction of this section? 


Solution: Let G be the function that assigns a grade to a student in our discrete mathematics class. 
Note that G(Adams) = A, for instance. The domain of G is the set {Adams, Chou, Goodfriend, 
Rodriguez, Stevens}, and the codomain is the set {A, B, C, D, F}. The range of G is the set 
{A, B, C, F}, because each grade except D is assigned to some student. < 


Let R be the relation with ordered pairs (Abdul, 22), (Brenda, 24), (Carla, 21), (Desire, 22), 
(Eddie, 24), and (Felicia, 22). Here each pair consists of a graduate student and this student’s 
age. Specify a function determined by this relation. 


Solution: Tf f is a function specified by R, then f(Abdul) = 22, f(Brenda) = 24, 
f (Carla) = 21, f (Desire) = 22, f (Eddie) = 24, and f (Felicia) = 22. (Here, f(x) is the age 
of x, where x is a student.) For the domain, we take the set {Abdul, Brenda, Carla, Desire, 
Eddie, Felicia}. We also need to specify a codomain, which needs to contain all possible ages 
of students. Because it is highly likely that all students are less than 100 years old, we can take 
the set of positive integers less than 100 as the codomain. (Note that we could choose a different 
codomain, such as the set of all positive integers or the set of positive integers between 10 and 
90, but that would change the function. Using this codomain will also allow us to extend the 
function by adding the names and ages of more students later.) The range of the function we 
have specified is the set of different ages of these students, which is the set {21, 22, 24}. < 


Let f be the function that assigns the last two bits of a bit string of length 2 or greater to that 
string. For example, f(11010) = 10. Then, the domain of / is the set of all bit strings of length 
2 or greater, and both the codomain and range are the set {00, 01, 10, 11}. < 


Let f:Z — Zassign the square of an integer to this integer. Then, f (x) = x7, where the domain 
of f is the set of all integers, the codomain of f is the set of all integers, and the range of f is 
the set of all integers that are perfect squares, namely, {0, 1, 4,9,...}. 


The domain and codomain of functions are often specified in programming languages. For 
instance, the Java statement 


int floor(float real){. . .} 
and the C++ function statement 
int function (float x){...} 


both tell us that the domain of the floor function is the set of real numbers (represented by 
floating point numbers) and its codomain is the set of integers. 


A function is called real-valued if its codomain is the set of real numbers, and it is called 
integer-valued if its codomain is the set of integers. Two real-valued functions or two integer- 
valued functions with the same domain can be added, as well as multiplied. 
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Let f; and f> be functions from A to R. Then f; + fo and f; fo are also functions from A 
to R defined for all x € A by 


(fi + fa)(*) = fi) + fa), 
(fi fa) = fi) fa). 


Note that the functions f; + fo and /) f2 have been defined by specifying their values at x in 
terms of the values of f; and fp at x. 


Let f; and f> be functions from R to R such that f,(x) = x? and fo(x) = x — x”. What are 
the functions f, + fo and f) fo? 


Solution: From the definition of the sum and product of functions, it follows that 
(fit AQ) = fie) + fA) =x? + @-x7) =x 
and 
(fi f2)(@) = x7 — x7) = x? — x4 4 


When f is a function from A to B, the image of a subset of A can also be defined. 


Let f be a function from A to B and let S be a subset of A. The image of S under the function 


f is the subset of B that consists of the images of the elements of S. We denote the image of 
S by f(S), so 


FS) = {tf |dseS @ = f(s))}. 


We also use the shorthand { f(s) | s € S} to denote this set. 


Remark: The notation f(S) for the image of the set S under the function f is potentially 
ambiguous. Here, f(S) denotes a set, and not the value of the function f for the set S. 


Let A = {a, b,c, d, e} and B = {1, 2,3, 4} with f(a) = 2, f(b) = 1, f(c) = 4, f(d) = 1, and 
f (e) = 1. The image of the subset § = {b, c, d} is the set f(S) = {1, 4}. < 


One-to-One and Onto Functions 
Some functions never assign the same value to two different domain elements. These functions 


are said to be one-to-one. 


A function f is said to be one-to-one, or an injunction, if and only if f (a) = f(b) implies that 
a = b for all a and b in the domain of f. A function is said to be injective if it is one-to-one. 
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FIGURE 3 A One-to-One Function. 


Note that a function f is one-to-one if and only if f(a) 4 f(b) whenever a 4 b. This way 
of expressing that f is one-to-one is obtained by taking the contrapositive of the implication in 
the definition. 


Remark: We can express that f is one-to-one using quantifiers as VaVb(f (a) = f(b) > a= b) 
or equivalently VaVb(a 4 b > f(a) # f(b)), where the universe of discourse is the domain 
of the function. 


We illustrate this concept by giving examples of functions that are one-to-one and other 
functions that are not one-to-one. 


Determine whether the function f from {a, b, c, d} to {1, 2,3, 4,5} with f(a) = 4, f(b) =5, 
f(c) = 1, and f(d) = 3 is one-to-one. 


Solution: The function f is one-to-one because f takes on different values at the four elements 
of its domain. This is illustrated in Figure 3. < 


Determine whether the function f(x) = x? from the set of integers to the set of integers is 
one-to-one. 


Solution: The function f(x) = x is not one-to-one because, for instance, fd) =f(i-dD= 1, 
but 1 4 —-1. 

Note that the function f(x) = x? with its domain restricted to Z*+ is one-to-one. (Techni- 
cally, when we restrict the domain of a function, we obtain a new function whose values agree 
with those of the original function for the elements of the restricted domain. The restricted 
function is not defined for elements of the original domain outside of the restricted domain.) 4 


Determine whether the function f(x) = x + | from the set of real numbers to itself is one-to- 
one. 


Solution: The function f(x) = x + 1 is a one-to-one function. To demonstrate this, note that 
x+1Ay+1whenx Fy. < 


Suppose that each worker in a group of employees is assigned a job from a set of possible 
jobs, each to be done by a single worker. In this situation, the function f that assigns a job 
to each worker is one-to-one. To see this, note that if x and y are two different workers, then 
f(x) # f(y) because the two workers x and y must be assigned different jobs. 


We now give some conditions that guarantee that a function is one-to-one. 
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FIGURE 4 An Onto Function. 


A function f whose domain and codomain are subsets of the set of real numbers is called 
increasing if f(x) < f(y), and strictly increasing if f(x) < f(y), whenever x < y and x 
and y are in the domain of f. Similarly, f is called decreasing if f(x) > f(y), and strictly 
decreasing if f (x) > f(y), whenever x < y and x and y are in the domain of f. (The word 
strictly in this definition indicates a strict inequality.) 


Remark: A function f is increasing if VxVy(x < y > f(x) < f(y)), strictly increasing if 
VxVy(x < y > f(x) < f(Q)), decreasing if VxVy(x < y > f(x) = f(y)), and strictly de- 
creasing if VxVy(x < y > f(x) > f(y)), where the universe of discourse is the domain of f. 


From these definitions, it can be shown (see Exercises 26 and 27) that a function that is 
either strictly increasing or strictly decreasing must be one-to-one. However, a function that is 
increasing, but not strictly increasing, or decreasing, but not strictly decreasing, is not one-to-one. 

For some functions the range and the codomain are equal. That is, every member of the 
codomain is the image of some element of the domain. Functions with this property are called 
onto functions. 


A function f from A to B is called onto, or a surjection, if and only if for every element 
b € B there is anelementa € A with f(a) = b.A function f is called surjective if it is onto. 


Remark: A function f is onto if Vyax(f (x) = y), where the domain for x is the domain of the 
function and the domain for y is the codomain of the function. 


We now give examples of onto functions and functions that are not onto. 


Let f be the function from {a, b,c, d} to {1, 2, 3} defined by f(a) = 3, f(b) = 2, fc) = 1, 
and f(d) = 3. Is f an onto function? 


Solution: Because all three elements of the codomain are images of elements in the domain, we 


see that f is onto. This is illustrated in Figure 4. Note that if the codomain were {1, 2, 3, 4}, 
then f would not be onto. 


Is the function f(x) = x? from the set of integers to the set of integers onto? 


Solution: The function f is not onto because there is no integer x withx? = —1,forinstance. 4 


Is the function f(x) = x + 1 from the set of integers to the set of integers onto? 
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FIGURE 5 Examples of Different Types of Correspondences. 
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Solution: This function is onto, because for every integer y there is an integer x such that 
f (x) = y. To see this, note that f(x) = y if and only if x + 1 = y, which holds if and only if 
x=y-l. < 


Consider the function f in Example 11 that assigns jobs to workers. The function /f is onto if 
for every job there is a worker assigned this job. The function f is not onto when there is at 
least one job that has no worker assigned it. < 


The function f is a one-to-one correspondence, or a bijection, if it is both one-to-one and 
onto. We also say that such a function is bijective. 


Examples 16 and 17 illustrate the concept of a bijection. 


Let f be the function from {a, b, c, d} to {1, 2,3, 4} with f(a) = 4, f(b) = 2, f(c) = 1, and 
f(@ = 3.Is f a bijection? 


Solution: The function f is one-to-one and onto. It is one-to-one because no two values in 
the domain are assigned the same function value. It is onto because all four elements of the 
codomain are images of elements in the domain. Hence, f is a bijection. < 


Figure 5 displays four functions where the first is one-to-one but not onto, the second is onto 
but not one-to-one, the third is both one-to-one and onto, and the fourth is neither one-to-one 
nor onto. The fifth correspondence in Figure 5 is not a function, because it sends an element to 
two different elements. 

Suppose that f is a function from a set A to itself. If A is finite, then f is one-to-one if and 
only if it is onto. (This follows from the result in Exercise 72.) This is not necessarily the case 
if A is infinite (as will be shown in Section 2.5). 


Let A be a set. The identity function on A is the function 14 : A > A, where 
ta(x) =x 


for all x € A. In other words, the identity function v4 is the function that assigns each element 
to itself. The function 14 is one-to-one and onto, so it is a bijection. (Note that is the Greek 
letter iota.) < 


For future reference, we summarize what needs be to shown to establish whether a function 
is one-to-one and whether it is onto. It is instructive to review Examples 8-17 in light of this 
summary. 
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Suppose that f: A > B. 


To show that f is injective Show that if f(x) = f(y) for arbitrary x, y € A with x # y, 
thenee—ay. 


To show that f is not injective Find particular elements x, y € A such that x # y and 
F(x) = fF). 

To show that f is surjective Consider an arbitrary element y € B and find an element x € A 
such that f(x) = y. 


To show that f is not surjective Find a particular y € B such that f(x) # y forall x € A. 


Inverse Functions and Compositions of Functions 


Now consider a one-to-one correspondence f from the set A to the set B. Because f is an onto 
function, every element of B is the image of some element in A. Furthermore, because f is also 
a one-to-one function, every element of B is the image of a unique element of A. Consequently, 
we can define a new function from B to A that reverses the correspondence given by /. This 
leads to Definition 9. 


Let f be a one-to-one correspondence from the set A to the set B. The inverse function of 
f is the function that assigns to an element b belonging to B the unique element a in A 
such that f(a) = b. The inverse function of f is denoted by f~!. Hence, f—!(b) = a when 
f(a) = b. 


Remark: Be sure not to confuse the function f—! with the function 1/f, which is the function 
that assigns to each x in the domain the value 1/f (x). Notice that the latter makes sense only 
when f (x) is a non-zero real number. 


Figure 6 illustrates the concept of an inverse function. 

If a function f is not a one-to-one correspondence, we cannot define an inverse function of 
f. When f is not a one-to-one correspondence, either it is not one-to-one or it is not onto. If 
f is not one-to-one, some element b in the codomain is the image of more than one element in 
the domain. If f is not onto, for some element b in the codomain, no element a in the domain 
exists for which f(a) = b. Consequently, if f is not a one-to-one correspondence, we cannot 
assign to each element b in the codomain a unique element a in the domain such that f(a) = b 
(because for some b there is either more than one such a or no such a). 

A one-to-one correspondence is called invertible because we can define an inverse of this 
function. A function is not invertible if it is not a one-to-one correspondence, because the 
inverse of such a function does not exist. 


f 1) 


FIGURE 6 The Function f~! Is the Inverse of Function f. 
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Let f be the function from {a, b, c} to {1, 2, 3} such that f(a) = 2, f(b) = 3, and f(c) = 1. 
Is f invertible, and if it is, what is its inverse? 


Solution: The function f is invertible because it is a one-to-one correspondence. The in- 
verse function f—! reverses the correspondence given by f, so f~'(1) =c, f~!(2) =a, and 


f'"Oy=h. 


Let f : Z— Zbe such that f(x) = x + 1. Is f invertible, and if it is, what is its inverse? 


Solution: The function f has an inverse because it is a one-to-one correspondence, as follows 
from Examples 10 and 14. To reverse the correspondence, suppose that y is the image of x, so 
that y = x + 1. Then x = y — 1. This means that y — | is the unique element of Z that is sent 
to y by f. Consequently, f~!(y) = y — 1. 


Let f be the function from R to R with f(x) = x?. Is f invertible? 


Solution: Because f (—2) = f (2) = 4, f is not one-to-one. If an inverse function were defined, 
it would have to assign two elements to 4. Hence, f is not invertible. (Note we can also show 
that f is not invertible because it is not onto.) < 


Sometimes we can restrict the domain or the codomain of a function, or both, to obtain an 
invertible function, as Example 21 illustrates. 


Show that if we restrict the function f(x) = x? in Example 20 to a function from the set of all 
nonnegative real numbers to the set of all nonnegative real numbers, then f is invertible. 


Solution: The function f(x) = x? from the set of nonnegative real numbers to the set of non- 
negative real numbers is one-to-one. To see this, note that if f(x) = f(y), then x? = y?, so 
x? — y* = (x + y)(x — y) = 0. This means that x + y = Oorx — y = 0,sox = —yorx =y. 
Because both x and y are nonnegative, we must have x = y. So, this function is one-to-one. 
Furthermore, f(x) = x? is onto when the codomain is the set of all nonnegative real numbers, 
because each nonnegative real number has a square root. That is, if y is a nonnegative real 
number, there exists a nonnegative real number x such that x = ./y, which means that x7 =y., 
Because the function f(x) = x* from the set of nonnegative real numbers to the set of non- 
negative real numbers is one-to-one and onto, it is invertible. Its inverse is given by the rule 


f'O) = V7. < 


Let g be a function from the set A to the set B and let f be a function from the set B to the 
set C. The composition of the functions f and g, denoted for all a € A by f 0 g, is defined 
by 


(f 0 g)(a) = f(g). 


In other words, f o g is the function that assigns to the element a of A the element assigned 
by f to g(a). That is, to find (f o g)(a) we first apply the function g to a to obtain g(a) and 
then we apply the function f to the result g(a) to obtain (f o g)(a) = f(g(a)). Note that the 
composition f o g cannot be defined unless the range of g is a subset of the domain of f. In 
Figure 7 the composition of functions is shown. 
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(f © g(a) 


F(g(a)) 


fog 
FIGURE 7 The Composition of the Functions f and g. 
Let g be the function from the set {a, b, c} to itself such that g(a) = b, g(b) = c, and g(c) =a. 


Let f be the function from the set {a, b, c} to the set {1, 2, 3} such that f(a) = 3, f(b) = 2, and 
f (c) = 1. What is the composition of f and g, and what is the composition of g and f? 


Solution: The composition fog is defined by (fog)(a)= f(g(a)) = f(b) =2, 


(f 0g) (b) = f(g)) = fF) = 1, and (f o g)(c) = f(g(©)) = f@ =3. 
Note that g o f is not defined, because the range of f is not a subset of the domain of g.4 


Let f and g be the functions from the set of integers to the set of integers defined by 
f(x) = 2x +3 and g(x) = 3x + 2. What is the composition of f and g? What is the com- 
position of g and f? 


Solution: Both the compositions f o g and g o f are defined. Moreover, 
(f 0 g(x) = f(g(x)) = f Bx +2) = 2x +2) 43 = 6x +7 
and 


(go f)(x) = g(f (x) = g(x +3) = 3(2x +3) 4+2=6x4+11. < 


Remark: Note that even though f og and go f are defined for the functions f and g in 
Example 23, f o g and g o f are not equal. In other words, the commutative law does not hold 
for the composition of functions. 


When the composition of a function and its inverse is formed, in either order, an identity 
function is obtained. To see this, suppose that f is a one-to-one correspondence from the set A 
to the set B. Then the inverse function f~! exists and is a one-to-one correspondence from B 
to A. The inverse function reverses the correspondence of the original function, so f~!(b) = a 
when f(a) = b, and f(a) = b when f—!(b) = a. Hence, 


(fo f\@ =f '¢@) =f! =a, 
and 
(fo f—')b) = f( f(b) = f@ =b. 


Consequently f~! o f =v, and f o f~! =v, where v4 and vg are the identity functions on 
the sets A and B, respectively. That is, (f~!)~! = f. 
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The Graphs of Functions 


We can associate a set of pairs in A x B to each function from A to B. This set of pairs is called 
the graph of the function and is often displayed pictorially to aid in understanding the behavior 
of the function. 


Let f be a function from the set A to the set B. The graph of the function f is the set of 
ordered pairs {(a, b) | a € A and f(a) = dD}. 


From the definition, the graph of a function f from A to B is the subset of A x B containing the 
ordered pairs with the second entry equal to the element of B assigned by f to the first entry. 
Also, note that the graph of a function f from A to B is the same as the relation from A to B 
determined by the function f, as described on page 139. 


Display the graph of the function f(n) = 2n + 1 from the set of integers to the set of integers. 
Solution: The graph of f is the set of ordered pairs of the form (n, 2n + 1), where n is an integer. 
This graph is displayed in Figure 8. < 
Display the graph of the function f(x) = x” from the set of integers to the set of integers. 
Solution: The graph of f is the set of ordered pairs of the form (x, f(x)) = (x, x”), where x is 
an integer. This graph is displayed in Figure 9. < 
Some Important Functions 

Next, we introduce two important functions in discrete mathematics, namely, the floor and ceiling 
functions. Let x be a real number. The floor function rounds x down to the closest integer less 
than or equal to x, and the ceiling function rounds x up to the closest integer greater than or 
equal to x. These functions are often used when objects are counted. They play an important 


role in the analysis of the number of steps used by procedures to solve problems of a particular 
size. 
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FIGURE 8 The Graph of FIGURE 9 The Graph of 


f (n) = 2n +1 from Z to Z. f (x) = x? from Z to Z. 
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The floor function assigns to the real number x the largest integer that is less than or equal to 
x. The value of the floor function at x is denoted by |x]. The ceiling function assigns to the 
real number x the smallest integer that is greater than or equal to x. The value of the ceiling 
function at x is denoted by [x]. 


Remark: The floor function is often also called the greatest integer function. It is often denoted 
by [x]. 


These are some values of the floor and ceiling functions: 


5) = 0,131 =1,1-3] =-1, [-41 =9, 13.1) =3, 13.11=4,17) =7,[71=7. < 


We display the graphs of the floor and ceiling functions in Figure 10. In Figure 10(a) we display 
the graph of the floor function |x|. Note that this function has the same value throughout the 
interval [n,n + 1), namely n, and then it jumps up ton + | when x =n + 1. In Figure 10(b) 
we display the graph of the ceiling function [x]. Note that this function has the same value 
throughout the interval (n,n + 1], namely n + 1, and then jumps to n + 2 when x is a little 
larger than n + 1. 

The floor and ceiling functions are useful in a wide variety of applications, including those 
involving data storage and data transmission. Consider Examples 27 and 28, typical of basic 
calculations done when database and data communications problems are studied. 


Data stored on a computer disk or transmitted over a data network are usually represented as a 
string of bytes. Each byte is made up of 8 bits. How many bytes are required to encode 100 bits 
of data? 


Solution: To determine the number of bytes needed, we determine the smallest integer that is at 
least as large as the quotient when 100 is divided by 8, the number of bits in a byte. Consequently, 
[100/8] = [12.5] = 13 bytes are required. < 


In asynchronous transfer mode (ATM) (a communications protocol used on backbone networks), 
data are organized into cells of 53 bytes. How many ATM cells can be transmitted in 1 minute 
over a connection that transmits data at the rate of 500 kilobits per second? 


Solution: In 1 minute, this connection can transmit 500,000 - 60 = 30,000,000 bits. Each ATM 
cell is 53 bytes long, which means that it is 53 - 8 = 424 bits long. To determine the number 
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=3. = = 1 2 3 jf. =f = 1 2 3 
718 oe 1 == 
eo—o-2 + Oo—e 2+ 
eo—o =3 5 += 


(a) y=[] (b) y=] 


FIGURE 10 Graphs of the (a) Floor and (b) Ceiling Functions. 
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EXAMPLE 29 


Extra 
Examples 


2 


TABLE 1 Useful Properties of the Floor 
and Ceiling Functions. 
(n is an integer, x is a real number) 


[x] =n if and only ifn <x <n 
[x] = nif and only ifn -—1 <x 


[x] =n if and only ifx-—1l<n 


[x] =n if and only ifx <n <x+1 
(2) x-1l< [x] <x< [x] <x4+1 


(3a) [—xJ] = —[x] 
(3b) [—x] = —Lx] 
(4a) [x +n] =[xJ]4 
(4b) [x +n] = [x] 


of cells that can be transmitted in 1 minute, we determine the largest integer not exceeding the 
quotient when 30,000,000 is divided by 424. Consequently, |30,000,000/424| = 70,754 ATM 
cells can be transmitted in | minute over a 500 kilobit per second connection. < 


Table 1, with x denoting a real number, displays some simple but important properties of the 
floor and ceiling functions. Because these functions appear so frequently in discrete mathematics, 
it is useful to look over these identities. Each property in this table can be established using the 
definitions of the floor and ceiling functions. Properties (1a), (1b), (1c), and (1d) follow directly 
from these definitions. For example, (1a) states that |x] = n if and only if the integer n is less 
than or equal to x and n + | is larger than x. This is precisely what it means for 7 to be the 
greatest integer not exceeding x, which is the definition of |x| =n. Properties (1b), (1c), and 
(1d) can be established similarly. We will prove property (4a) using a direct proof. 


Proof: Suppose that |x] = m, where m is a positive integer. By property (1a), it follows that 
m <x <m-+1.Adding v to all three quantities in this chain of two inequalities shows that m + 
n<x+n<m-+n-+ 1. Using property (1a) again, we see that |x +n] =m+n=|[x]+n. 
This completes the proof. Proofs of the other properties are left as exercises. | 


The floor and ceiling functions enjoy many other useful properties besides those displayed in 
Table 1. There are also many statements about these functions that may appear to be correct, but 
actually are not. We will consider statements about the floor and ceiling functions in Examples 29 
and 30. 

A useful approach for considering statements about the floor function is to let x =n +e, 
where n = [x] is an integer, and e¢, the fractional part of x, satisfies the inequality 0 < € < 1. 
Similarly, when considering statements about the ceiling function, it is useful to write x = n — €, 
where n = [x] is an integer and 0 < € < I. 


Prove that if x is a real number, then [2x] = |x| + Lx + 5]. 


Solution: To prove this statement we let x = n + €, where n is an integer and 0 < € < |. There 
are two cases to consider, depending on whether « is less than, or greater than or equal to 5 
(The reason we choose these two cases will be made clear in the proof.) 
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We first consider the case when 0 < € < 5 In this case, 2x = 2n + 2¢€ and |2x| = 2n 


because 0 < 2e€ < 1. Similarly, x + 5 =n+ G +e€),so|x+ al =n, because 0 < 5 +e<l. 


Consequently, [2x | = 2n and |x] + Lx + 51 =n+n=2n. 

Next, we consider the case when 5 <e<1. In this case, 2x =2n+2€= 
(2n+ 1)+ (e-— 1). Because 0 < 2e —1 <1, it follows that [2x| =2n+ 1. Because 
Ix +5] = [n+ (5 +6)] =|[n+1+(e- 5) and 0 <e€ — 5 < |, it follows that [x +5] = 
n+ 1. Consequently, [2x | = 2n+ 1 and [x] + Lx + 5| =n+(n+1) =2n+ 1. This con- 
cludes the proof. < 


EXAMPLE 30 Prove or disprove that [x + y] = [x] + [y] for all real numbers x and y. 


Solution: Although this statement may appear reasonable, it is false. A counterexample is sup- 
plied by x = 5 and y = 5: With these values we find that [x + y] = [5 + 51 = [1] = 1, but 


[x] + fy] = 1414 4] =141=2. < 


There are certain types of functions that will be used throughout the text. These include 
polynomial, logarithmic, and exponential functions. A brief review of the properties of these 
functions needed in this text is given in Appendix 2. In this book the notation log x will be used 
to denote the logarithm to the base 2 of x, because 2 is the base that we will usually use for 
logarithms. We will denote logarithms to the base b, where b is any real number greater than 1, 
by log, x, and the natural logarithm by In x. 

Another function we will use throughout this text is the factorial function f: N— Z™, 
denoted by f(n) = n!. The value of f(n) = n! is the product of the first n positive integers, so 
f(a) =1-2---(7—1)-n [and f(0) = 0! = 1). 


EXAMPLE 31 We have f(1)=l!=1, f(2)=2!=1-2=2, f(@)=6!=1-2-3-4-5-6= 720, 
and f(20) = 1:2:3+4-5-6-7-8-9-10-11: 12+ 13-14-15-16-17-18-19-20= 
2,432,902,008, 176,640,000. 4 


Example 31 illustrates that the factorial function grows extremely rapidly as n grows. 
The rapid growth of the factorial function is made clearer by Stirling’s formula, a result from 
higher mathematics that tell us that n! ~ /27n(n/e)". Here, we have used the notation f (n) ~ 
g(n), which means that the ratio f(n)/g(n) approaches | as n grows without bound (that is, 
limp—+oo f (n)/g(n) = 1). The symbol ~ is read “is asymptotic to.” Stirling’s formula is named 
Links after James Stirling, a Scottish mathematician of the eighteenth century. 


JAMES STIRLING (1692-1770) James Stirling was born near the town of Stirling, Scotland. His family strongly supported the 
Jacobite cause of the Stuarts as an alternative to the British crown. The first information known about James is that he entered Balliol 
College, Oxford, on a scholarship in 1711. However, he later lost his scholarship when he refused to pledge his allegiance to the 
British crown. The first Jacobean rebellion took place in 1715, and Stirling was accused of communicating with rebels. He was 
charged with cursing King George, but he was acquitted of these charges. Even though he could not graduate from Oxford because 
of his politics, he remained there for several years. Stirling published his first work, which extended Newton’s work on plane curves, 
in 1717. He traveled to Venice, where a chair of mathematics had been promised to him, an appointment that unfortunately fell 
through. Nevertheless, Stirling stayed in Venice, continuing his mathematical work. He attended the University of Padua in 1721, 
and in 1722 he returned to Glasgow. Stirling apparently fled Italy after learning the secrets of the Italian glass industry, avoiding the 
efforts of Italian glass makers to assassinate him to protect their secrets. 

In late 1724 Stirling moved to London, staying there 10 years teaching mathematics and actively engaging in research. In 1730 
he published Methodus Differentialis, his most important work, presenting results on infinite series, summations, interpolation, and 
quadrature. It is in this book that his asymptotic formula for n! appears. Stirling also worked on gravitation and the shape of the 
earth; he stated, but did not prove, that the earth is an oblate spheroid. Stirling returned to Scotland in 1735, when he was appointed 
manager of a Scottish mining company. He was very successful in this role and even published a paper on the ventilation of mine 
shafts. He continued his mathematical research, but at a reduced pace, during his years in the mining industry. Stirling is also noted 
for surveying the River Clyde with the goal of creating a series of locks to make it navigable. In 1752 the citizens of Glasgow 
presented him with a silver teakettle as a reward for this work. 
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DEFINITION 13 


EXAMPLE 32 


Exercises 


1. Why is f not a function from R to R if 


a) f(x) =1/x? 
b) f(x) = Jx? 


°) fx) =tJV024+1? 


2. Determine whether f is a function from Z to R if 


a) f(n) =n. 


b) f(a) =Vn2 +1. 
ce) f(n) =1/@? -4). 

3. Determine whether f is a function from the set of all bit c) 
strings to the set of integers if 


Partial Functions 


A program designed to evaluate a function may not produce the correct value of the function for 
all elements in the domain of this function. For example, a program may not produce a correct 
value because evaluating the function may lead to an infinite loop or an overflow. Similarly, in 
abstract mathematics, we often want to discuss functions that are defined only for a subset of 
the real numbers, such as 1/x, ./x, and arcsin (x). Also, we may want to use such notions as 
the “youngest child” function, which is undefined for a couple having no children, or the “time 
of sunrise,” which is undefined for some days above the Arctic Circle. To study such situations, 
we use the concept of a partial function. 


A partial function f from a set A to a set B is an assignment to each element a in a subset 
of A, called the domain of definition of f, of a unique element b in B. The sets A and B are 
called the domain and codomain of f, respectively. We say that f is undefined for elements 
in A that are not in the domain of definition of f. When the domain of definition of f equals 
A, we say that f is a total function. 


Remark: We write f : A — B to denote that f is a partial function from A to B. Note that 
this is the same notation as is used for functions. The context in which the notation is used 
determines whether /f is a partial function or a total function. 


The function f : Z > Rwhere f (n) = /nisa partial function from Z to R where the domain of 
definition is the set of nonnegative integers. Note that f is undefined for negative integers. <4 


5. Find the domain and range of these functions. Note that 
in each case, to find the domain, determine the set of 
elements assigned values by the function. 

a) the function that assigns to each bit string the number 
of ones in the string minus the number of zeros in the 
string 

b) the function that assigns to each bit string twice the 
number of zeros in that string 


the function that assigns the number of bits left over 
when a bit string is split into bytes (which are blocks 


a) f(S) is the position of a0 bitin S. 

b) f(S) is the number of 1 bits in S. 

c) f(S) is the smallest integer 7 such that the ith bit of 
Sis 1 and f(S) =0 when S is the empty string, the 
string with no bits. 

. Find the domain and range of these functions. Note that 

in each case, to find the domain, determine the set of 

elements assigned values by the function. 

a) the function that assigns to each nonnegative integer 
its last digit 

b) the function that assigns the next largest integer to a 
positive integer 

c) the function that assigns to a bit string the number of 
one bits in the string 

d) the function that assigns to a bit string the number of 
bits in the string 


of 8 bits) 


d) the function that assigns to each positive integer the 
largest perfect square not exceeding this integer 


. Find the domain and range of these functions. 


a) the function that assigns to each pair of positive inte- 
gers the first integer of the pair 


b) the function that assigns to each positive integer its 
largest decimal digit 


c) the function that assigns to a bit string the number of 
ones minus the number of zeros in the string 


d) the function that assigns to each positive integer the 
largest integer not exceeding the square root of the 
integer 


e) the function that assigns to a bit string the longest 
string of ones in the string 


7. 


10. 


11. 
12. 


13. 
14. 


15. 


16. 


17. 


Find the domain and range of these functions. 

a) the function that assigns to each pair of positive inte- 
gers the maximum of these two integers 

b) the function that assigns to each positive integer the 
number of the digits 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 that do 
not appear as decimal digits of the integer 

c) the function that assigns to a bit string the number of 
times the block 11 appears 

d) the function that assigns to a bit string the numerical 
position of the first 1 in the string and that assigns the 
value 0 to a bit string consisting of all Os 


. Find these values. 


a) [1.1] b) [1.1] 

ce) |-0.1] d) [—0.1] 

e) [2.99] f) [—2.99] 

g) lot lot h) (13) +1514 51 
. Find these values. 

a) [7] b) Lg! 

e) [-7] d) L-31 

e) [3] f) |-1] 

g) +131! h) L5- 3) 


Determine whether each of these functions from 
{a, b, c, d} to itself is one-to-one. 

a) f(a) =), f(b) =a, fic) =c, f(d)=d 

b) fa@ =), fb) =b, fi) =d, f@=c 

ce) f(a) =d, f(b) =), floc) =c, fd) =d 

Which functions in Exercise 10 are onto? 

Determine whether each of these functions from Z to Z 
is one-to-one. 

a) f(n)=n—-1 b) f(y) =n? +1 

:) fan d) f(a) = [n/2] 

Which functions in Exercise 12 are onto? 

Determine whether f: Z x Z — Zis onto if 

a) f(m,n) = 2m —n. 

b) f(m,n) =m? —n?. 

c) f(m,n)=m+n+1. 

d) f(m,n) = |m| — |n|. 

e) f(m,n) =m? —4. 

Determine whether the function f: Z x Z — Z is onto 
if 

a) f(m,n)=m-+n. 

b) f(m,n) =m? +n’. 

c) f(m,n) =m. 

d) f(m,n) = |n|. 

e) f(m,n)=m—n. 

Consider these functions from the set of students in a 
discrete mathematics class. Under what conditions is the 
function one-to-one if it assigns to a student his or her 
a) mobile phone number. 

b) student identification number. 

c) final grade in the class. 

d) home town. 

Consider these functions from the set of teachers in a 
school. Under what conditions is the function one-to-one 
if it assigns to a teacher his or her 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


27. 


28. 
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a) office. 

b) assigned bus to chaperone in a group of buses taking 
students on a field trip. 

c) salary. 

d) social security number. 

Specify a codomain for each of the functions in Exercise 

16. Under what conditions is each of these functions with 

the codomain you specified onto? 

Specify a codomain for each of the functions in Exercise 

17. Under what conditions is each of the functions with 

the codomain you specified onto? 

Give an example of a function from N to N that is 

a) one-to-one but not onto. 

b) onto but not one-to-one. 

c) both onto and one-to-one (but different from the iden- 
tity function). 

d) neither one-to-one nor onto. 

Give an explicit formula for a function from the set of 

integers to the set of positive integers that is 

a) one-to-one, but not onto. 

b) onto, but not one-to-one. 

c) one-to-one and onto. 

d) neither one-to-one nor onto. 

Determine whether each of these functions is a bijection 

from R to R. 

a) f(x) =—-3x+4 

b) f(x) = —3x7 +7 

ce) f@%)=@+)D/@ +2) 

d) fx) =x° +1 

Determine whether each of these functions is a bijection 

from R to R. 

a) f(x) =2x4+1 

b) f@) =x? +1 

) fax 

d) f(x) = 02+ 1/2 +2) 

Let f:R — R and let f(x) > 0 for all x € R. Show 

that f(x) is strictly increasing if and only if the func- 

tion g(x) = 1/f (x) is strictly decreasing. 

Let f:R — R and let f(x) > 0 for all x € R. Show 

that f(x) is strictly decreasing if and only if the func- 

tion g(x) = 1/f (x) is strictly increasing. 


a) Prove that a strictly increasing function from R to it- 
self is one-to-one. 

b) Give an example of an increasing function from R to 
itself that is not one-to-one. 


a) Prove that a strictly decreasing function from R to 
itself is one-to-one. 

b) Give an example of a decreasing function from R to 
itself that is not one-to-one. 


Show that the function f(x) = e* from the set of real 
numbers to the set of real numbers is not invertible, but 
if the codomain is restricted to the set of positive real 
numbers, the resulting function is invertible. 
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29. 


30. 


31. 


32. 


33. 


* 34, 


* 35. 


36. 


37. 


38. 


39. 


40. 


41. 


Show that the function f(x) = |x| from the set of real 
numbers to the set of nonnegative real numbers is not 
invertible, but if the domain is restricted to the set of non- 
negative real numbers, the resulting function is invertible. 
Let S = {—1, 0, 2, 4, 7}. Find f(S) if 

a) fx)=1. b) f(x) =2x4+1. 

ce) f(x) = [x/5]. d) f(x) =L@? + 1)/3]. 
Let f(x) = [x?/3]. Find f(S) if 

a) S = {—2, —-1,0, 1, 2, 3}. 

b) S = {0, 1, 2, 3, 4, 5}. 

ce) S= {1,5,7, 11}. 

d) S = {2, 6, 10, 14}. 

Let f(x) = 2x where the domain is the set of real num- 
bers. What is 


a) f(Z)? b) f(N)? c) f(R)? 

Suppose that g is a function from A to B and f is a 

function from B to C. 

a) Show that if both f and g are one-to-one functions, 
then f o g is also one-to-one. 

b) Show that if both f and g are onto functions, then 
f og is also onto. 


If f and f og are one-to-one, does it follow that g is 

one-to-one? Justify your answer. 

If f and f o g are onto, does it follow that g is onto? 

Justify your answer. 

Find f og and go f, where f(x) =x? +1 and g(x) = 

x +2, are functions from R to R. 

Find f + g and fg for the functions f and g given in 

Exercise 36. 

Let f(x) = ax +b and g(x) = cx +d, where a, b,c, 

and d are constants. Determine necessary and suffi- 

cient conditions on the constants a,b,c, and d so that 

fogs=sgof. 

Show that the function f(x) = ax + b from R to R is 

invertible, where a and b are constants, with a 4 0, and 

find the inverse of f. 

Let f be a function from the set A to the set B. Let S and 

T be subsets of A. Show that 

a) f(SUT) = f(S)U f(T). 

b) f(SOT) CS fF(S)N f(T). 

a) Give an example to show that the inclusion in part (b) 
in Exercise 40 may be proper. 

b) Show that if f is one-to-one, the inclusion in part (b) 
in Exercise 40 is an equality. 


Let f be a function from the set A to the set B. Let S be a 
subset of B. We define the inverse image of S to be the subset 
of A whose elements are precisely all pre-images of all ele- 
ments of S. We denote the inverse image of S by f—!(S), so 
f—(S) ={a€ A| f(a) € S}. (Beware: The notation f~! is 
used in two different ways. Do not confuse the notation intro- 
duced here with the notation f—!(y) for the value at y of the 


inverse of the invertible function f. Notice also that f—!(S), 
the inverse image of the set S, makes sense for all functions f, 
not just invertible functions.) 


42. 


43. 


44, 


45. 


46. 


47. 


48. 


49. 


50. 


51. 


52. 


53. 


54. 


55. 


56. 


57. 


58. 


Let f be the function from R to R defined by 
f(x) =x”. Find 

a) f~'({1)). 

c) f~'({x |x > 4). 
Let g(x) = Lx]. Find 

a) g~'({0}). 

ce) g(x |0<.x <1)). 
Let f be a function from A to B. Let S and T be subsets 
of B. Show that 

a) f-\(SUT) = f-\(S)U f'(7). 

b) f-'SAT) =f (S)N f"(7). 

Let f be a function from A to B. Let S be a subset of B. 
Show that f—!(S) = f-1(S). 

Show that |x + 5! is the closest integer to the number x, 


except when x is midway between two integers, when it 
is the larger of these two integers. 


b) f-'x|0<-x <1)). 


b) g“'({-1,0, 1). 


Show that [x — 5] is the closest integer to the number x, 
except when x is midway between two integers, when it 
is the smaller of these two integers. 


Show that if x is a real number, then [x] — |x] = 1 if x 
is not an integer and [x] — |x] = O if x is an integer. 
Show that if x is areal number, then x — 1 < |x| <x < 
[x] <x+1. 

Show that if x is a real number and m is an integer, then 
[x +m] = [x] +m. 

Show that if x is a real number and n is an integer, then 
a) x <nifand only if |x] <n. 

b) n < x if and only ifn < [x]. 


Show that if x is a real number and n is an integer, then 
a) x <nifand only if [x] <n. 
b) n < x if and only ifn < [x]. 


Prove that if n is an integer, then [n/2| = n/2ifnis even 
and (n — 1)/2 ifn is odd. 

Prove that if x is a real number, then |—x ] = —[x] and 
[-x] = —-Lx]. 

The function INT is found on some calculators, where 
INT(x) = |x] when x is a nonnegative real number and 
INT(x) = [x] when x is a negative real number. Show 
that this INT function satisfies the identity INT(—x) = 
—INT(x). 

Let a and b be real numbers with a < b. Use the floor 
and/or ceiling functions to express the number of inte- 
gers n that satisfy the inequality a <n < b. 

Let a and b be real numbers with a < b. Use the floor 
and/or ceiling functions to express the number of inte- 
gers n that satisfy the inequality a <n <b. 


How many bytes are required to encode n bits of data 
where n equals 
a) 4? b) 10? 


c) 500? d) 3000? 


59. 


60. 


61. 


62. 


63. 


64. 


65. 


66. 


67. 


68. 


69. 


70. 


71. 


How many bytes are required to encode n bits of data 
where n equals 

a) 7? b) 17? c) 1001? d) 28,800? 
How many ATM cells (described in Example 28) can be 
transmitted in 10 seconds over a link operating at the fol- 
lowing rates? 

a) 128 kilobits per second (1 kilobit = 1000 bits) 

b) 300 kilobits per second 

c) 1 megabit per second (1 megabit = 1,000,000 bits) 
Data are transmitted over a particular Ethernet network 
in blocks of 1500 octets (blocks of 8 bits). How many 
blocks are required to transmit the following amounts of 
data over this Ethernet network? (Note that a byte is a 
synonym for an octet, a kilobyte is 1000 bytes, and a 
megabyte is 1,000,000 bytes.) 

a) 150 kilobytes of data 

b) 384 kilobytes of data 

c) 1.544 megabytes of data 

d) 45.3 megabytes of data 


Draw the graph of the function f(n) = 1 —n? from Z 
to Z. 


Draw the graph of the function f(x) = [2x] from R 
toR. 


Draw the graph of the function f(x) = |x/2]| from R 
toR. 


Draw the graph of the function f(x) = |x] + |x/2] from 
RtoR. 


Draw the graph of the function f(x) = [x] + Lx/2] from 
RtoR. 


Draw graphs of each of these functions. 


a) f(x)=|x+4I b) f(x) = [2x41] 

c) f(x) = [x/3] d) f(x) = [1/x] 

e) f(x) =[x-—2]+ |x +2] 

f) f(x) = [2x] [x/2] g) f(x) =[lx- 5 I + 5] 


Draw graphs of each of these functions. 


a) f(x) = [3x —2] b) f(x) = [0.2x] 
ce) f@)=l|-I1/x] d) f(x) = |x? 

e) f(x) = [x/2]Lx/2]  f) f(x) = Lx/2] + [x/2] 
g) f(x) = |2[x/21+ 5] 


Find the inverse function of f(x) = x3 +1. 

Suppose that f is an invertible function from Y to Z 
and g is an invertible function from X to Y. Show 
that the inverse of the composition f o g is given by 
(fogyt=gtof 

Let S be a subset of a universal set U. The characteristic 
function fs of S is the function from U to the set {0, 1} 
such that fs(x) = 1 if x belongs to S and fs(x) = Oif x 
does not belong to S. Let A and B be sets. Show that for 
allx € U, 

a) fane) = fax): fa) 

b) faus(x) = fax) + fax) — fax): fa) 

ce) fx) =1— fa) 

d) faos) = fae) + fa(x) — 2 fa) fa (x) 


c= 72, 


73. 


74. 


75. 


76. 


77. 


78. 


c=" 79, 


* 80. 
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Suppose that f is a function from A to B, where A and B 

are finite sets with |A| = |B]. Show that f is one-to-one 

if and only if it is onto. 

Prove or disprove each of these statements about the floor 

and ceiling functions. 

a) [|x|] = Lx] for all real numbers x. 

b) [2x] = 2[x] whenever x is a real number. 

ce) [x] + fy] — [x + y] = 0 or 1 whenever x and y are 
real numbers. 

d) [xy] = [x] [9] for all real numbers x and y. 


) [=] x+1 
e _— = 

2 2 
Prove or disprove each of these statements about the floor 
and ceiling functions. 
a) | [x] ] = [x] for all real numbers x. 
b) [x + y] = Lx] + Ly] for all real numbers x and y. 
ec) [ [x/2] /2] = [x/4] for all real numbers x. 
d) |./Tx] | = |/x | for all positive real numbers x. 


e) Lx] +bLyJ+l¢+y] < [2x]4+[|2y] for all real 
numbers x and y. 


for all real numbers x. 


Prove that if x is a positive real number, then 


a) LVixJ | = lve]. 

b) [/TxT1 = [./*1. 

Let x be a real number. 

IxJ+ Let 5) +le+ 3). 

For each of these partial functions, determine its domain, 

codomain, domain of definition, and the set of values for 

which it is undefined. Also, determine whether it is a total 

function. 

a) f:Z>R,f@=1/n 

b) f:Z— Z, f(a) = [n/2] 

ce) f:Z2xZ—-Q, f(m,n) =m/n 

d) f:ZxZ— Z, f(m,n) =mn 

e) f:Z2xZ—Z, f(m,n)=m—-nifm>n 

a) Show thata partial function from A to B can be viewed 
as a function f* from A to B U {u}, where u is not an 
element of B and 


Show that [3x] = 


f (a) if a belongs to the domain 
of definition of f 
u if f is undefined at a. 


f(@= 


b) Using the construction in (a), find the function f* 
corresponding to each partial function in Exercise 77. 


a) Show that if a set S has cardinality m, where m is a 
positive integer, then there is a one-to-one correspon- 
dence between S and the set {1,2,..., m}. 

b) Show that if S and T are two sets each with m ele- 
ments, where m is a positive integer, then there is a 
one-to-one correspondence between S and T. 

Show that a set S is infinite if and only if there is a proper 

subset A of S such that there is a one-to-one correspon- 

dence between A and S. 
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Sequences and Summations 


DEFINITION 1 


EXAMPLE 1 


Introduction 


Sequences are ordered lists of elements, used in discrete mathematics in many ways. For ex- 
ample, they can be used to represent solutions to certain counting problems, as we will see in 
Chapter 8. They are also an important data structure in computer science. We will often need 
to work with sums of terms of sequences in our study of discrete mathematics. This section 
reviews the use of summation notation, basic properties of summations, and formulas for the 
sums of terms of some particular types of sequences. 

The terms of a sequence can be specified by providing a formula for each term of the 
sequence. In this section we describe another way to specify the terms of a sequence using 
a recurrence relation, which expresses each term as a combination of the previous terms. We 
will introduce one method, known as iteration, for finding a closed formula for the terms of a 
sequence specified via a recurrence relation. Identifying a sequence when the first few terms 
are provided is a useful skill when solving problems in discrete mathematics. We will provide 
some tips, including a useful tool on the Web, for doing so. 


Sequences 


A sequence is a discrete structure used to represent an ordered list. For example, 1, 2, 3, 5, 8 is 
a sequence with five terms and 1, 3,9, 27, 81,..., 3”, ... is an infinite sequence. 


A sequence 1s a function from a subset of the set of integers (usually either the set {0, 1, 2, .. .} 
or the set {1, 2, 3, .. .}) toa set S. We use the notation a, to denote the image of the integer n. 
We call a, a term of the sequence. 


We use the notation {a,,} to describe the sequence. (Note that a, represents an individual 
term of the sequence {a,}. Be aware that the notation {a,} for a sequence conflicts with the 
notation for a set. However, the context in which we use this notation will always make it clear 
when we are dealing with sets and when we are dealing with sequences. Moreover, although we 
have used the letter a in the notation for a sequence, other letters or expressions may be used 
depending on the sequence under consideration. That is, the choice of the letter a is arbitrary.) 

We describe sequences by listing the terms of the sequence in order of increasing subscripts. 


Consider the sequence {a,}, where 
1 
an =. 
n 
The list of the terms of this sequence, beginning with a), namely, 
a1, 42, 43, 44,..., 
starts with 


1 111 
ar aa 


DEFINITION 2 


EXAMPLE 2 


DEFINITION 3 


EXAMPLE 3 


EXAMPLE 4 
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A geometric progression is a sequence of the form 


REL SIET iT Be ee 


where the initial term a and the common ratio r are real numbers. 


Remark: A geometric progression is a discrete analogue of the exponential function f(x) = 


ar*. 


The sequences {b,,} with b, = (—1)", {cn} with cy = 2-5”, and {d,} with d, = 6- (1/3)” are 
geometric progressions with initial term and common ratio equal to | and —1; 2 and 5; and 6 
and 1/3, respectively, if we start at n = 0. The list of terms bo, b1, b2, b3, ba, ... begins with 


1,-—1,1,-1,1,...; 
the list of terms co, ci, C2, C3, C4, ... begins with 
2, 10, 50, 250, 1250, ...; 
and the list of terms do, d;, do, d3, d4, ... begins with 


a P 


2,55 x5 meee 
: "3° 9" 27 


An arithmetic progression is a sequence of the form 
ad,a+d,a+2d,...,a+nd,... 


where the initial term a and the common difference d are real numbers. 


Remark: An arithmetic progression is a discrete analogue of the linear function f(x) = dx +a. 


The sequences {s,} with s, = —1 + 4n and {t,} with ¢, = 7 — 3n are both arithmetic progres- 
sions with initial terms and common differences equal to —1 and 4, and 7 and —3, respectively, 
if we start at n = 0. The list of terms so, 51, 52, 53, ... begins with 
=1,.3, 7, Mec sy 
and the list of terms fo, f1, f2, £3, ... begins with 
TAA, S2 ccs < 
Sequences of the form aj, a2,...,@, are often used in computer science. These finite 


sequences are also called strings. This string is also denoted by aja2...dy. (Recall that bit 
strings, which are finite sequences of bits, were introduced in Section 1.1.) The length of a 
string is the number of terms in this string. The empty string, denoted by A, is the string that 
has no terms. The empty string has length zero. 


The string abcd is a string of length four. < 
Recurrence Relations 


In Examples 1—3 we specified sequences by providing explicit formulas for their terms. There 
are many other ways to specify a sequence. For example, another way to specify a sequence is 
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DEFINITION 4 


EXAMPLE 5 


EXAMPLE 6 


Hop along to Chapter 8 
to learn how to find a 
formula for the Fibonacci 
numbers. 


DEFINITION 5 


Links 


EXAMPLE 7 


to provide one or more initial terms together with a rule for determining subsequent terms from 
those that precede them. 


A recurrence relation for the sequence {ay} is an equation that expresses a, in terms of one or 
more of the previous terms of the sequence, namely, ag, a1, ... , dn—1, for all integers n with 
n > no, where ng is a nonnegative integer. A sequence is called a solution of a recurrence 
relation if its terms satisfy the recurrence relation. (A recurrence relation is said to recursively 
define a sequence. We will explain this alternative terminology in Chapter 5.) 


Let {a,} be a sequence that satisfies the recurrence relation a, = d,—; +3 forn = 1,2,3,..., 
and suppose that ag = 2. What are aj, a2, and a3? 


Solution: We see from the recurrence relation that aj = a9 + 3 = 24+3=5. It then follows 
thata2 =5+3=8anda3 =8+3=11. <{ 


Let {a,} be a sequence that satisfies the recurrence relation a, = dy,—1| — an—2 for n= 
2,3,4,..., and suppose that ag = 3 and a, = 5. What are az and a3? 


Solution: We see from the recurrence relation that az = aj — dg = 5 — 3 = 2 and a3 = a2 — 
a, = 2—5 = —3. We can find aa, a5, and each successive term in a similar way. < 


The initial conditions for a recursively defined sequence specify the terms that precede the 
first term where the recurrence relation takes effect. For instance, the initial condition in Example 
5 is dg = 2, and the initial conditions in Example 6 are ag = 3 and a; = 5. Using mathematical 
induction, a proof technique introduced in Chapter 5, it can be shown that a recurrence relation 
together with its initial conditions determines a unique solution. 

Next, we define a particularly useful sequence defined by a recurrence relation, known as 
the Fibonacci sequence, after the Italian mathematician Fibonacci who was born in the 12th 
century (see Chapter 5 for his biography). We will study this sequence in depth in Chapters 5 
and 8, where we will see why it is important for many applications, including modeling the 
population growth of rabbits. 


The Fibonacci sequence, fo, fi, f2, ..., 18 defined by the initial conditions fo = 0, f; = 1, 
and the recurrence relation 


Sn = fn—-1 at ip 


TOT See eee 


Find the Fibonacci numbers fo, f3, fa, fs, and fe. 


Solution: The recurrence relation for the Fibonacci sequence tells us that we find successive 
terms by adding the previous two terms. Because the initial conditions tell us that fo = 0 and 
J\ = 1, using the recurrence relation in the definition we find that 


f=fitfo=Hl+o0=), 
B=frtfi=14+1=2, 
fa= fat fo=24+15=3, 
fs = fat fg =34+255, < 
fo = fst fa=5+3=8. 


EXAMPLE 8 


EXAMPLE 9 


EXAMPLE 10 
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Suppose that {a,} is the sequence of integers defined by a, = n!, the value of the factorial 
function at the integer n, where n = 1, 2,3,.... Because n! = n((n — 1)(n—2)...2-D= 
n(n — 1)! =nap_1, we see that the sequence of factorials satisfies the recurrence relation 
An = Ndy—1, together with the initial condition a; = 1. 


We say that we have solved the recurrence relation together with the initial conditions when 
we find an explicit formula, called a closed formula, for the terms of the sequence. 


Determine whether the sequence {a,}, where a, = 3n for every nonnegative integer n, is a 
solution of the recurrence relation a, = 2d,_1 — Qn—2 forn = 2,3,4,... . Answer the same 
question where a, = 2” and where a, = 5. 


Solution: Suppose that a, = 3n for every nonnegative integer n. Then, for n > 2, we see that 
2dn—1 — An—2 = 2(3(n — 1)) — 3(n — 2) = 3n = ay. Therefore, {a,}, where a, = 3n, 1s a so- 
lution of the recurrence relation. 

Suppose that a, = 2” for every nonnegative integer n. Note thatag = 1,a; = 2,andaz = 4. 
Because 2a, — a9 = 2-2—1=3 #4), we see that {a,}, where a, = 2”, is not a solution of 
the recurrence relation. 

Suppose that a, = 5 for every nonnegative integer n. Then for n > 2, we see that a, = 
2dn—| — Qn-2 = 2-5 —5 = 5 = ay. Therefore, {a,}, where a, = 5, is a solution of the recur- 
rence relation. 


Many methods have been developed for solving recurrence relations. Here, we will introduce 
a straightforward method known as iteration via several examples. In Chapter 8 we will study 
recurrence relations in depth. In that chapter we will show how recurrence relations can be used 
to solve counting problems and we will introduce several powerful methods that can be used to 
solve many different recurrence relations. 


Solve the recurrence relation and initial condition in Example 5. 


Solution: We can successively apply the recurrence relation in Example 5, starting with the 
initial condition a; = 2, and working upward until we reach a,, to deduce a closed formula for 
the sequence. We see that 


a2 =2+3 
a3 = (24+3)+3=2+3-2 
ag = (24+2-3)4+3=2+3-3 


Qn = On-1+3=(24+3-(#—2))4+3=24+3—-)). 
We can also successively apply the recurrence relation in Example 5, starting with the 


term a, and working downward until we reach the initial condition a, = 2 to deduce this same 
formula. The steps are 


an = An-1 +3 
= (Qnh_-2 +3) +3 =Q,_2+3-2 
= (4-3 +3) +3-2 =ayn_-34+3-3 


=a, + 3(n — 2) = (a, +3) 4+ 3(n — 2) = 24+3(n — 1). 
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EXAMPLE 11 


Extra 
Examples 
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At each iteration of the recurrence relation, we obtain the next term in the sequence by 
adding 3 to the previous term. We obtain the nth term after n — | iterations of the recurrence 
relation. Hence, we have added 3(n — 1) to the initial term ap = 2 to obtain a,. This gives us 
the closed formula a, = 2 + 3(n — 1). Note that this sequence is an arithmetic progression. 4 


The technique used in Example 10 is called iteration. We have iterated, or repeatedly used, 
the recurrence relation. The first approach is called forward substitution — we found successive 
terms beginning with the initial condition and ending with a,. The second approach is called 
backward substitution, because we began with a, and iterated to express it in terms of falling 
terms of the sequence until we found it in terms of a;. Note that when we use iteration, we 
essential guess a formula for the terms of the sequence. To prove that our guess is correct, we 
need to use mathematical induction, a technique we discuss in Chapter 5. 

In Chapter 8 we will show that recurrence relations can be used to model a wide variety of 
problems. We provide one such example here, showing how to use a recurrence relation to find 
compound interest. 


Compound Interest Suppose that a person deposits $10,000 in a savings account at a bank 
yielding 11% per year with interest compounded annually. How much will be in the account 
after 30 years? 


Solution: To solve this problem, let P,, denote the amount in the account after n years. Because 
the amount in the account after n years equals the amount in the account after n — 1 years plus 
interest for the nth year, we see that the sequence { P,,} satisfies the recurrence relation 


P, = Pa—1 +0.11Py—1 = (1.11) Pr-1- 


The initial condition is Pp = 10,000. 
We can use an iterative approach to find a formula for P,,. Note that 


P, = (1.11) Po 
Py = (1.11) P; = (1.11)? Pp 
Ps = (1.11) Py = (1.11)? Po 


P, = (1.11) Pa—1 = (1.11)" Pp. 


When we insert the initial condition Po = 10,000, the formula P,, = (1.11)"10,000 is obtained. 
Inserting n = 30 into the formula P, = (1.11)”10,000 shows that after 30 years the account 
contains 


P39 = (1.11)°°10,000 = $228,922.97. 4 


Special Integer Sequences 


A common problem in discrete mathematics is finding a closed formula, a recurrence relation, 
or some other type of general rule for constructing the terms of a sequence. Sometimes only a 
few terms of a sequence solving a problem are known; the goal is to identify the sequence. Even 
though the initial terms of a sequence do not determine the entire sequence (after all, there are 
infinitely many different sequences that start with any finite set of initial terms), knowing the 
first few terms may help you make an educated conjecture about the identity of your sequence. 
Once you have made this conjecture, you can try to verify that you have the correct sequence. 


EXAMPLE 12 


Extra po 
Examples 
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EXAMPLE 13 


EXAMPLE 14 


EXAMPLE 15 
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When trying to deduce a possible formula, recurrence relation, or some other type of rule 
for the terms of a sequence when given the initial terms, try to find a pattern in these terms. You 
might also see whether you can determine how a term might have been produced from those 
preceding it. There are many questions you could ask, but some of the more useful are: 


@ Are there runs of the same value? That is, does the same value occur many times in a 
row? 

® Are terms obtained from previous terms by adding the same amount or an amount that 
depends on the position in the sequence? 

® Are terms obtained from previous terms by multiplying by a particular amount? 

® Are terms obtained by combining previous terms in a certain way? 

@ Are there cycles among the terms? 


Find formulae for the sequences with the following first five terms: (a) 1, 1/2, 1/4, 1/8, 1/16 
(b) 1, 3,5, 7,9 (c)1,—1,1,-1,1. 


Solution: (a) We recognize that the denominators are powers of 2. The sequence witha, = 1/2”, 
n = 0, 1, 2, ... is a possible match. This proposed sequence is a geometric progression with 
a=landr=1/2. 

(b) We note that each term is obtained by adding 2 to the previous term. The sequence 
witha, = 2n+ 1,n =0, 1, 2,...1s a possible match. This proposed sequence is an arithmetic 
progression with a = | andd = 2. 

(c) The terms alternate between | and —1. The sequence with a, = (—1)",n =0, 1,2... 
is a possible match. This proposed sequence is a geometric progression witha = | andr = —1. 


Examples 13-15 illustrate how we can analyze sequences to find how the terms are con- 
structed. 


How can we produce the terms of a sequence if the first 10 terms are 1, 2, 2, 3, 3, 3, 4, 4, 4, 4? 


Solution: In this sequence, the integer | appears once, the integer 2 appears twice, the integer 3 
appears three times, and the integer 4 appears four times. A reasonable rule for generating this 
sequence is that the integer n appears exactly n times, so the next five terms of the sequence 
would all be 5, the following six terms would all be 6, and so on. The sequence generated this 
way is a possible match. < 


How can we produce the terms of a sequence if the first 10 terms are 5, 11, 17, 23, 29, 35, 41, 
47, 53,59? 


Solution: Note that each of the first 10 terms of this sequence after the first is obtained by adding 
6 to the previous term. (We could see this by noticing that the difference between consecutive 
terms is 6.) Consequently, the nth term could be produced by starting with 5 and adding 6 a 
total of n — | times; that is, a reasonable guess is that the nth term is 5 + 6(m — 1) = 6n — 1. 
(This is an arithmetic progression with a = 5 andd = 6.) 


How can we produce the terms of a sequence if the first 10 terms are 1, 3, 4, 7, 11, 18, 29, 47, 
76, 123? 


Solution: Observe that each successive term of this sequence, starting with the third term, 
is the sum of the two previous terms. That is, 4=3+1,7=4+3, 11 =7+4, and so on. 
Consequently, if L,, is the nth term of this sequence, we guess that the sequence is determined 
by the recurrence relation L, = Lyj—, + Ly—2 with initial conditions L; = | and Lz = 3 (the 
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EXAMPLE 16 


Check out the puzzles at 


the OEIS site. 


Links 


TABLE 1 Some Useful Sequences. 


First 10 Terms 


1,4, 9, 16, 25, 36, 49, 64, 81, 100,... 
1, 8, 27, 64, 125, 216, 343, 512, 729, 1000, ... 


1, 16, 81, 256, 625, 1296, 2401, 4096, 6561, 10000, ... 
2,4, 8, 16, 32, 64, 128, 256, 512, 1024, ... 

3, 9, 27, 81, 243, 729, 2187, 6561, 19683, 59049, ... 

1, 2, 6, 24, 120, 720, 5040, 40320, 362880, 3628800, ... 
1, 1,2, 3,5, 8, 13,21, 34,55, 89,... 


same recurrence relation as the Fibonacci sequence, but with different initial conditions). This 
sequence is known as the Lucas sequence, after the French mathematician Francois Edouard 
Lucas. Lucas studied this sequence and the Fibonacci sequence in the nineteenth century. <4 


Another useful technique for finding a rule for generating the terms of a sequence is to 
compare the terms of a sequence of interest with the terms of a well-known integer sequence, 
such as terms of an arithmetic progression, terms of a geometric progression, perfect squares, 
perfect cubes, and so on. The first 10 terms of some sequences you may want to keep in mind 
are displayed in Table 1. 


Conjecture a simple formula for a, if the first 10 terms of the sequence {a,} are 1, 7, 25, 79, 
241, 727, 2185, 6559, 19681, 59047. 


Solution: To attack this problem, we begin by looking at the difference of consecutive terms, 
but we do not see a pattern. When we form the ratio of consecutive terms to see whether each 
term is a multiple of the previous term, we find that this ratio, although not a constant, is close 
to 3. So it is reasonable to suspect that the terms of this sequence are generated by a formula 
involving 3”. Comparing these terms with the corresponding terms of the sequence {3”}, we 
notice that the nth term is 2 less than the corresponding power of 3. We see that a, = 3” — 2 
for 1 <n < 10 and conjecture that this formula holds for all n. < 


We will see throughout this text that integer sequences appear in a wide range of contexts in 
discrete mathematics. Sequences we have encountered or will encounter include the sequence 
of prime numbers (Chapter 4), the number of ways to order n discrete objects (Chapter 6), the 
number of moves required to solve the famous Tower of Hanoi puzzle with n disks (Chapter 8), 
and the number of rabbits on an island after n months (Chapter 8). 

Integer sequences appear in an amazingly wide range of subject areas besides discrete 
mathematics, including biology, engineering, chemistry, and physics, as well as in puzzles. An 
amazing database of over 200,000 different integer sequences can be found in the On-Line 
Encyclopedia of Integer Sequences (OEIS). This database was originated by Neil Sloane in the 
1960s. The last printed version of this database was published in 1995 ({[SIPI95]); the current 
encyclopedia would occupy more than 750 volumes of the size of the 1995 book with more than 
10,000 new submissions a year. There is also a program accessible via the Web that you can use 
to find sequences from the encyclopedia that match initial terms you provide. 


Summations 


Next, we consider the addition of the terms of a sequence. For this we introduce summation 
notation. We begin by describing the notation used to express the sum of the terms 


Am, 4m+1,+++54n 


EXAMPLE 17 


Extra 
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@ 
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from the sequence {a,}. We use the notation 


n 
» n 
aj ? ya aj ’ or Pein aj 


j=m 
(read as the sum from j = m to j =n of a;) to represent 
Am + Am+1 +++++ an. 


Here, the variable j is called the index of summation, and the choice of the letter j as the 
variable is arbitrary; that is, we could have used any other letter, such as 7 or k. Or, in notation, 


n n n 
Ya=Da= Daw 


j=m i=m k=m 


Here, the index of summation runs through all integers starting with its lower limit m and ending 
with its upper limit 7. A large uppercase Greek letter sigma, 5~, is used to denote summation. 

The usual laws for arithmetic apply to summations. For example, when a and Db are real 
numbers, we have Vai Gx; + byj) =a at xj+ byi=1 yj, where x1, x2,...,Xn and 
Y1, 2, +++ 5 Yn are real numbers. (We do not present a formal proof of this identity here. Such a 
proof can be constructed using mathematical induction, a proof method we introduce in Chap- 
ter 5. The proof also uses the commutative and associative laws for addition and the distributive 
law of multiplication over addition.) 

We give some examples of summation notation. 


Use summation notation to express the sum of the first 100 terms of the sequence {a;}, where 
a; =1/j for j = 1,2, 3,.... 


Solution: The lower limit for the index of summation is |, and the upper limit is 100. We write 
this sum as 


NEIL SLOANE (BORN 1939) Neil Sloane studied mathematics and electrical engineering at the Uni- 
versity of Melbourne on a scholarship from the Australian state telephone company. He mastered many 
telephone-related jobs, such as erecting telephone poles, in his summer work. After graduating, he designed 
minimal-cost telephone networks in Australia. In 1962 he came to the United States and studied electri- 
cal engineering at Cornell University. His Ph.D. thesis was on what are now called neural networks. He 
took a job at Bell Labs in 1969, working in many areas, including network design, coding theory, and 
sphere packing. He now works for AT&T Labs, moving there from Bell Labs when AT&T split up in 
1996. One of his favorite problems is the kissing problem (a name he coined), which asks how many 
spheres can be arranged in n dimensions so that they all touch a central sphere of the same size. (In two 


dimensions the answer is 6, because 6 pennies can be placed so that they touch a central penny. In three dimensions, 12 billiard 
balls can be placed so that they touch a central billiard ball. Two billiard balls that just touch are said to “kiss,” giving rise to the 
terminology “kissing problem” and “kissing number.”) Sloane, together with Andrew Odlyzko, showed that in 8 and 24 dimensions, 
the optimal kissing numbers are, respectively, 240 and 196,560. The kissing number is known in dimensions 1, 2, 3, 4, 8, and 24, but 
not in any other dimensions. Sloane’s books include Sphere Packings, Lattices and Groups, 3d ed., with John Conway; The Theory 
of Error-Correcting Codes with Jessie MacWilliams; The Encyclopedia of Integer Sequences with Simon Plouffe (which has grown 
into the famous OEIS website); and The Rock-Climbing Guide to New Jersey Crags with Paul Nick. The last book demonstrates his 
interest in rock climbing; it includes more than 50 climbing sites in New Jersey. 
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EXAMPLE 19 


EXAMPLE 20 
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THEOREM 1 


What is the value of 4 72 


Solution: We have 


5 
YoPaPePrPr "4s 
j=l 
=1+44+9+16+25 < 
=55. 
What is the value of se 4(-1)*? 


Solution: We have 


8 
Yip = (- bt + C8 + (- Do + (74+ C8 
k=4 
=1+(-D+1+(-1)+1 < 


Sometimes it - useful to shift the index of summation in a sum. This is often done when 
two sums need to be added but their indices of summation do not match. When shifting an index 
of summation, it is important to make the appropriate changes in the corresponding summand. 
This is illustrated by Example 20. 

Suppose we have the sum 


but want the index of summation to run between 0 and 4 rather than from 1 to 5. To do this, 
we let k = j — 1. Then the new summation index runs from 0 (because k = 1 — 0 = 0 when 
J = 1) to 4 (because k = 5 — 1 = 4 when j = 5), and the term 7" becomes (k + 1)”. Hence, 


It is easily checked that both sums are 1 +4+9-+4 16425 =55. < 


Sums of terms of geometric progressions commonly arise (such sums are called geometric 
series). Theorem | gives us a formula for the sum of terms of a geometric progression. 


If a and r are real numbers and r + 0, then 


py re 


ul fe 
Sar r—1 
j=0 


(n+ l)a fit? = Il. 


nei ee II 


Proof: Let 


n 
Sn S ar’. 
j=0 
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To compute S, first multiply both sides of the equality by r and then manipulate the resulting 
sum as follows: 


rS; =r > ar) substituting summation formula for S 
j=0 
n 
= ari*! b istributi 
= y the distributive property 
j=0 
n+1 
= > ar* shifting the index of summation, with k = j + 1 
k=1 


n 
= ys or) + (ar"t! — a) removing k =n + | term and adding k = 0 term 


= Sy, + (ar" el a) substituting S for summation formula 


From these equalities, we see that 
rSy = Sn + (ar"*! — a). 


Solving for S, shows that if r € 1, then 


ar'+! _ q 
eae r=1 
If r = 1, then the S, = )°}o ari = in0 4 = (n+ Ia. 4 


EXAMPLE 21 Double summations arise in many contexts (as in the analysis of nested loops in computer 
programs). An example of a double summation is 


4 3 
dil. 
i=) j=1 
To evaluate the double sum, first expand the inner summation and then continue by computing 


the outer summation: 


4 3 


4 
pies: =) G+ 2i +31) 
i=1 j=1 i=l 
4 
=> 6 


i=1 4 
=64+124+18+4+24= 60. 


We can also use summation notation to add all values of a function, or terms of an indexed 
set, where the index of summation runs over all values in a set. That is, we write 


Yo f) 
ses 


to represent the sum of the values f(s), for all members s of S. 
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EXAMPLE 22 


EXAMPLE 23 


TABLE 2 Some Useful Summation Formulae. 


Closed Form 


P —a 
a“ rAl 
r-—1 . 


n(n + 1) 
2 


n(n + 1)(2n + 1) 
6 


n2(n + 1)? 


What is the value of >, . 0,2,4} 5? 


Solution: Because >, < {0,2,4} § fepresents the sum of the values of s for all the members of the 
set {0, 2, 4}, it follows that 


ye s=04244=6. a 
s €{0,2,4} 


Certain sums arise repeatedly throughout discrete mathematics. Having a collection of 
formulae for such sums can be useful; Table 2 provides a small table of formulae for commonly 
occurring sums. 

We derived the first formula in this table in Theorem 1. The next three formulae give us the 
sum of the first n positive integers, the sum of their squares, and the sum of their cubes. These 
three formulae can be derived in many different ways (for example, see Exercises 37 and 38). 
Also note that each of these formulae, once known, can easily be proved using mathematical 
induction, the subject of Section 5.1. The last two formulae in the table involve infinite series 
and will be discussed shortly. 

Example 23 illustrates how the formulae in Table 2 can be useful. 


Find 7) k2. 


Solution: First note that because pare = re e+ yar. we have 


100 100 49 
a ae eee 
k=50 k=1 k=1 


Using the formula ye ae = n(n + 1)(2n + 1)/6 from Table 2 (and proved in Exercise 38), 
we see that 


se 100-101-201 49-50-99 


6 6 


= 338,350 — 40,425 = 297,925. 4 
k=50 
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SOME INFINITE SERIES Although most of the summations in this book are finite sums, 
infinite series are important in some parts of discrete mathematics. Infinite series are usually 
studied in a course in calculus and even the definition of these series requires the use of calculus, 
but sometimes they arise in discrete mathematics, because discrete mathematics deals with infi- 
nite collections of discrete elements. In particular, in our future studies in discrete mathematics, 


we will find the closed forms for the infinite series in Examples 24 and 25 to be quite useful. 


EXAMPLE 24 (Requires calculus) Let x be a real number with |x| < 1. Find )°°°_ 9.x”. 
; kt =| 
et Solution: By Theorem 1 with a = 1 and r = x we see that )°), _g x” = a Because 
ra p> x= 
Examples 2 |x| < 1, x**+! approaches 0 as k approaches infinity. It follows that 
on a HT AI 1 
> x" = lim — = ‘ < 
k>o x-—1 x—1 L= x 

n=0 


We can produce new summation formulae by differentiating or integrating existing formulae. 


EXAMPLE 25 (Requires calculus) Differentiating both sides of the equation 


as 1 
k_ 
aS ~ J-x’ 


from Example 24 we find that 


Ce 
Sxtie 1 
el (1 — x)? 


(This differentiation is valid for |x| < 1 by a theorem about infinite series.) < 


Exercises 


. Find these terms of the sequence {a,}, where ay, = 

2-(—3)" +5". 

a) ao b) a, Cc) a4 d) as 

. What is the term ag of the sequence {a,} if a, equals 

a) 2"-1? b) 72 

c) 1+(-1)"? d) —(—2)"? 

. What are the terms ao, a1, a2, and a3 of the sequence {ay}, 

where a, equals 

a) 2”+1? b) (n+ 1)"t!?2 

ec) [n/2]? d) [n/2] + [n/2]? 

. What are the terms ao, a1, a2, and a3 of the sequence {ay}, 

where a, equals 

a) (—2)"? b) 3? 

c) 7+ 4"? d) 2” + (—2)"? 

. List the first 10 terms of each of these sequences. 

a) the sequence that begins with 2 and in which each 
successive term is 3 more than the preceding term 

b) the sequence that lists each positive integer three 
times, in increasing order 

c) the sequence that lists the odd positive integers in in- 
creasing order, listing each odd integer twice 


d) the sequence whose nth term is n! — 2” 


e) the sequence that begins with 3, where each succeed- 
ing term is twice the preceding term 


f) the sequence whose first term is 2, second term is 4, 
and each succeeding term is the sum of the two pre- 
ceding terms 


g) the sequence whose nth term is the number of bits 
in the binary expansion of the number n (defined in 
Section 4.2) 


h) the sequence where the nth term is the number of let- 
ters in the English word for the index n 


. List the first 10 terms of each of these sequences. 


a) the sequence obtained by starting with 10 and obtain- 
ing each term by subtracting 3 from the previous term 


b) the sequence whose nth term is the sum of the first n 
positive integers 

c) the sequence whose nth term is 3” — 2” 

d) the sequence whose nth term is |./n] 

e) the sequence whose first two terms are 1 and 5 and 
each succeeding term is the sum of the two previous 
terms 
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10. 


11. 


12. 


13. 


14. 


15. 


f) the sequence whose nth term is the largest integer 
whose binary expansion (defined in Section 4.2) has 
n bits (Write your answer in decimal notation.) 

g) the sequence whose terms are constructed sequen- 
tially as follows: start with 1, then add 1, then multiply 
by 1, then add 2, then multiply by 2, and so on 

h) the sequence whose nth term is the largest integer k 
such that k! <n 


. Find at least three different sequences beginning with the 


terms 1, 2, 4 whose terms are generated by a simple for- 
mula or rule. 


. Find at least three different sequences beginning with the 


terms 3, 5, 7 whose terms are generated by a simple for- 
mula or rule. 


. Find the first five terms of the sequence defined by each 


of these recurrence relations and initial conditions. 


a) dy = 6ay_1, a9 = 2 
b) ay, ae ii =) 
C) An = 4n—1 + 3an—2, a9 = 1, a, = 2 
d) ay = nay itn? ay 2,49 =1aq,=1 


€) dy = Ay—1 + An—3, 49 = 1, a) = 2, a2 = 0 


Find the first six terms of the sequence defined by each 
of these recurrence relations and initial conditions. 

a) dy = —2an_1,da9 = —1 

b) an = an—1 — Gn—2, AQ = 2, a, = —1 

C) dh = 3a?_,,a9 =1 

d) a) =Nnan—, + ae ag = —l,a, =0 

©) Gn = An—1 — Gn—-2 +Gn-3, 40 = 1, a) = 1,az = 2 


Let a, = 2” + 5-3" forn =0,1,2,.... 

a) Find ao, a1, a2, a3, and a4. 

b) Show that az = 5a, — 6ao, a3 = San 
a4 = 5a3 — 6a2. 

c) Show that a, = 5a,—| — 6a,—2 for all integers n with 
n> 2. 


6a,, and 


Show that the sequence {a,,} is a solution of the recurrence 
relation a, = —3day,—1 + 4ay_2 if 

a) a, = 0. b) a, = 1. 

c) an = (-4)”. qd) ap = 2(—4)" + 3. 


Is the sequence {a,,} a solution of the recurrence relation 
An = 8dy—1 — 16ay—2 if 


a) dn = 0? b) an = 1? 

C) a, = 2"? d) a, =4"? 

e) dy, =n4"? f) a, =2-4" +3n4"? 

8) dy) = (—-4)"? h) a, = 774"? 

For each of these sequences find a recurrence relation 


satisfied by this sequence. (The answers are not unique 
because there are infinitely many different recurrence 
relations satisfied by any sequence.) 


a) ad, =3 b) a, = 2n 
Cc) dy =2n+3 d) a, = 5” 
e) dy =n? f) an =n2+n 
2) dy =n+(-1)” h) a, =n! 


Show that the sequence {a,,} is a solution of the recurrence 
relation dy = An—1 + 2dn—2 + 2n — 9 if 

a) dy, = —n+2. 

b) a, = 5(-1)" —n +2. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


Cc) dy = 3(-1)" +2” —n +2. 

d) dy =7-2" —n+2. 

Find the solution to each of these recurrence relations with 
the given initial conditions. Use an iterative approach such 
as that used in Example 10. 

a) dn = —ay—1,49 =5 

b) dy = 4n-1 +3,a9 = 1 

C) Gn =an-1 —N,ag = 4 

d) a, = 2a,_1 —3,a9 = —-1 

e€) ay = (n+ I)an_-1, 49 = 2 

f) a, = 2nay_1,a9 = 3 

2) dn = —an-1 +n—1,a9 =7 

Find the solution to each of these recurrence relations and 
initial conditions. Use an iterative approach such as that 
used in Example 10. 


a) dn = 3an—1, a0 = 2 

D) an = an-1 + 2, ag = 3 

C) Qh =4,-1 +N, ag = 1 

d) dy = an-1 +20 +3,a9 = 4 

e€) dy = 2an-1 — 1,ag = 1 

f) dy =3a,-, + 1,a9 = 1 

£) Gn =Nay—1, a9 =5 

h) dy = 2nady_1,a9 = 1 

A person deposits $1000 in an account that yields 9% 

interest compounded annually. 

a) Set up a recurrence relation for the amount in the ac- 
count at the end of n years. 

b) Find an explicit formula for the amount in the account 
at the end of n years. 

c) How much money will the account contain after 100 
years? 

Suppose that the number of bacteria in a colony triples 

every hour. 

a) Setup arecurrence relation for the number of bacteria 
after n hours have elapsed. 

b) If 100 bacteria are used to begin a new colony, how 
many bacteria will be in the colony in 10 hours? 

Assume that the population of the world in 2010 was 6.9 

billion and is growing at the rate of 1.1% a year. 

a) Set up a recurrence relation for the population of the 
world n years after 2010. 

b) Find an explicit formula for the population of the 
world n years after 2010. 

c) What will the population of the world be in 2030? 

A factory makes custom sports cars at an increasing rate. 

In the first month only one car is made, in the second 

month two cars are made, and so on, with n cars made in 

the nth month. 

a) Set up a recurrence relation for the number of cars 
produced in the first n months by this factory. 

b) How many cars are produced in the first year? 

c) Find an explicit formula for the number of cars pro- 
duced in the first n months by this factory. 

An employee joined a company in 2009 with a starting 

salary of $50,000. Every year this employee receives a 

raise of $1000 plus 5% of the salary of the previous year. 


23. 


25. 


26. 


** 27, 


* 28. 


29. 


a) Set up a recurrence relation for the salary of this em- 
ployee n years after 2009. 

b) What will the salary of this employee be in 2017? 

c) Find an explicit formula for the salary of this em- 
ployee n years after 2009. 

Find a recurrence relation for the balance B(k) owed at 

the end of k months on a loan of $5000 at a rate of 7% 

if a payment of $100 is made each month. [Hint: Ex- 

press B(x) in terms of B(k — 1); the monthly interest is 

(0.07/12) B(k — 1).] 


. a) Findarecurrence relation for the balance B(k) owed at 


the end of k months on a loan at arate of r if a payment 
P is made on the loan each month. [Hint: Express 
Bk) in terms of B(k — 1) and note that the monthly 
interest rate is r/12.] 

b) Determine what the monthly payment P should be so 
that the loan is paid off after T months. 

For each of these lists of integers, provide a simple for- 

mula or rule that generates the terms of an integer se- 

quence that begins with the given list. Assuming that your 

formula or rule is correct, determine the next three terms 

of the sequence. 

a) 1,0,1,1,0,0,1,1,1,0,0,0,1,... 

b) 1,2, 2,3,4,4,5, 6, 6, 7, 8,8 

c) 1,0,2,0,4,0, 8,0, 16,0,... 

d) 3,6, 12, 24, 48, 96, 192,... 

e) 15,8, 1, —6, —13, —20, —27,... 

f) 3,5, 8, 12, 17, 23, 30, 38, 47,... 

g) 2, 16,54, 128, 250, 432, 686, . 

h) 2, 3,7, 25, 121, 721, 5041, 40321, . 

For each of these lists of integers, avid a simple for- 

mula or rule that generates the terms of an integer se- 

quence that begins with the given list. Assuming that your 

formula or rule is correct, determine the next three terms 

of the sequence. 

a) 3,6, 11, 18, 27, 38, 51, 66, 83, 102,... 

b) 7, 11, 15, 19, 23, 27, 31, 35, 39, 43,... 


c) 1,10, 11, 100, 101, 110, 111, 1000, 1001, 1010, 1011,... 


d) 1,2, 2, 2,3, 3,3,3,3,5,5,5,5,5,5,5,... 

e) 0, 2, 8, 26, 80, 242, 728, 2186, 6560, 19682, ... 

f) 1,3, 15, 105, 945, 10395, 135135, 2027025, 
34459425,... 

g) 1,0,0,1,1,1,0,0,0,0,1,1,1,1,1,... 

h) 2, 4, 16, 256, 65536, 4294967296, ... 

Show that if a, denotes the nth positive integer that is not 

a perfect square, then a, = n + {./n}, where {x} denotes 

the integer closest to the real number x. 

Let a, be the nth term of the sequence 1, 2, 2,3,3,3,4,4,4, 

4,5,5,5,5,5, 6, 6, 6, 6, 6,6, ..., constructed by including 

the integer k exactly k times. Show that a, = |./2n + 5]. 

What are the values of these a 


5 
a) Do (k+ 1) b) E(- —2) 


k=1 


d) > (2i+l = 2/) 
j=0 


10 
ce) >> 3 


i=1 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


37. 


* 38. 


39. 
40. 
*41, 


* 42, 
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Whatare the values of these sums, where S = {1, 3, 5, 7}? 


a) Di ae i 
jes jes 

ce) » d//) d) > 1 
jes jes 


What is the value of each of these sums of terms of a 
geometric progression? 
8 . Bo 8, 
a) >> 3-2/ b) >> 2/ 
j=0 j=l 
8 . 8 : 
ce) >> (-3)/ d) >° 2-(-3)/ 
j=2 j=0 


Find the value of each of these sums. 


8 a. : 
a) ) 1+ (-1)/) b) >) G/ -2/) 
j=0 j=0 
©. YY 2-3743-2) ad Yt! -2/) 
j=0 j=0 
Compute each of these double sums. 
2 3 2 3 
a) Ve dVG+sh) by) > 2 Cr 3y) 
i=1j=1 i=0j=0 
3. 2 2¢ 3 
od vi d dv dij 
i=1j=0 i=0j=1 
Compute each of these double sums. 


3 2 
a >» YG -ZV) b) > ¥ Gi +2/) 


pa ae 
) 2 : j d) 3 yi? 

i=1j=0 i=0j;=0 
Show that = (aj — 4j-1) = an — a0, where 
ao, 41,...,@, iS a sequence of real numbers. This type 


of sum is called telescoping. 

Use the identity 1/(k(kK + 1)) = 1/k —1/(kK+ 1) and 
Exercise 35 to compute }°y_ , 1/(k(k + 1)). 

Sum both sides of the identity k? — (k — 1)? =2k—-1 
from k = 1 tok =n and use Exercise 35 to find 


a) a formula for ~~ — (2k — 1) (the sum of the first n 
odd natural numbers). 


b) a formula for )°y_ |, k. 

Use the technique given in Exercise 35, together with the 
result of Exercise 37b, to derive the formula for yy =] k 
given in Table 2. [Hint: Take a, = k° in the telescoping 
sum in Exercise 35.] 


Find >,” jook- (Use Table 2.) 
Find 70 99>. (Use Table 2.) 


Find a formula for > alval. when m is a positive 
integer. 


Find a formula for )7'_ 4 
integer. 


[J/k|, when m is a positive 


There is also a special notation for products. The product of 


Am, 4m+15+++5 


n 
dy is represented by [| aj, read as the prod- 
j=m 


uct from j = m to j =n of aj. 


170 2/Basic Structures: Sets, Functions, Sequences, Sums, and Matrices 


43. What are the values of the following products? 44. Express n! using product notation. 
a) TTi2o/ b) TTi=si 
©) T2n! d) 2,2 45. Find 3-4 _ ji. 


Recall that the value of the factorial function at a positive in- 
teger n, denoted by n!, is the product of the positive integers 


from 1 to n, inclusive. Also, we specify that 0! = 1. 46. Find I = ji. 
Cardinality of Sets 


Introduction 


In Definition 4 of Section 2.1 we defined the cardinality of a finite set as the number of elements 
in the set. We use the cardinalities of finite sets to tell us when they have the same size, or when 
one is bigger than the other. In this section we extend this notion to infinite sets. That is, we will 
define what it means for two infinite sets to have the same cardinality, providing us with a way 
to measure the relative sizes of infinite sets. 

We will be particularly interested in countably infinite sets, which are sets with the same 
cardinality as the set of positive integers. We will establish the surprising result that the set of 
rational numbers is countably infinite. We will also provide an example of an uncountable set 
when we show that the set of real numbers is not countable. 

The concepts developed in this section have important applications to computer science. A 
function is called uncomputable if no computer program can be written to find all its values, 
even with unlimited time and memory. We will use the concepts in this section to explain why 
uncomputable functions exist. 

We now define what it means for two sets to have the same size, or cardinality. In Section 2.1, 
we discussed the cardinality of finite sets and we defined the size, or cardinality, of such sets. In 
Exercise 79 of Section 2.3 we showed that there is a one-to-one correspondence between any 
two finite sets with the same number of elements. We use this observation to extend the concept 
of cardinality to all sets, both finite and infinite. 


DEFINITION 1 _ Thesets A and B have the same cardinality if and only if there is a one-to-one correspondence 
from A to B. When A and B have the same cardinality, we write |A| = |B|. 


For infinite sets the definition of cardinality provides a relative measure of the sizes of two sets, 
rather than a measure of the size of one particular set. We can also define what it means for one 
set to have a smaller cardinality than another set. 


DEFINITION 2 If there is a one-to-one function from A to B, the cardinality of A is less than or the same as 
the cardinality of B and we write |A| < |B|. Moreover, when |A| < |B| and A and B have 
different cardinality, we say that the cardinality of A is less than the cardinality of B and we 
write |A| < |B|. 


Countable Sets 


We will now split infinite sets into two groups, those with the same cardinality as the set of 
natural numbers and those with a different cardinality. 


DEFINITION 3 


EXAMPLE 1 


You can always get a room 
at Hilbert’s Grand Hotel! 


Links 


Links 
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FIGURE 1 A One-to-One Correspondence Between Z* and the Set of Odd Positive 
Integers. 


A set that is either finite or has the same cardinality as the set of positive integers is called 
countable. A set that is not countable is called uncountable. When an infinite set S is countable, 
we denote the cardinality of S by So (where ® is aleph, the first letter of the Hebrew alphabet). 
We write |S| = Xo and say that S has cardinality “aleph null.” 


We illustrate how to show a set is countable in the next example. 


Show that the set of odd positive integers is a countable set. 


Solution: To show that the set of odd positive integers is countable, we will exhibit a one-to-one 
correspondence between this set and the set of positive integers. Consider the function 


f(n) =2n-1 


from Z* to the set of odd positive integers. We show that f is a one-to-one correspondence by 
showing that it is both one-to-one and onto. To see that it is one-to-one, suppose that f(n) = 
f(m). Then 2n — 1 = 2m — 1, son = m. To see that it is onto, suppose that f is an odd positive 
integer. Then ¢ is | less than an even integer 2k, where k is a natural number. Hence t = 2k — 1 = 
J (k). We display this one-to-one correspondence in Figure 1. < 


An infinite set is countable if and only if it is possible to list the elements of the set in a 
sequence (indexed by the positive integers). The reason for this is that a one-to-one correspon- 
dence f from the set of positive integers to a set S can be expressed in terms of a sequence 
a1, 42,.--,d,-.., where aj = f(1), a2 = f(2),...,aq, = f(n),.... 


HILBERT’S GRAND HOTEL We now describe a paradox that shows that something impos- 
sible with finite sets may be possible with infinite sets. The famous mathematician David Hilbert 
invented the notion of the Grand Hotel, which has a countably infinite number of rooms, each 
occupied by a guest. When a new guest arrives at a hotel with a finite number of rooms, and 
all rooms are occupied, this guest cannot be accommodated without evicting a current guest. 
However, we can always accommodate a new guest at the Grand Hotel, even when all rooms 
are already occupied, as we show in Example 2. Exercises 5 and 8 ask you to show that we can 
accommodate a finite number of new guests and a countable number of new guests, respectively, 
at the fully occupied Grand Hotel. 


DAVID HILBERT (1862-1943) — Hilbert, born in KGnigsberg, the city famous in mathematics for its seven 
bridges, was the son of a judge. During his tenure at Gottingen University, from 1892 to 1930, he made many 
fundamental contributions to a wide range of mathematical subjects. He almost always worked on one area of 
mathematics at a time, making important contributions, then moving to a new mathematical subject. Some areas 
in which Hilbert worked are the calculus of variations, geometry, algebra, number theory, logic, and mathematical 
physics. Besides his many outstanding original contributions, Hilbert is remembered for his famous list of 23 
difficult problems. He described these problems at the 1900 International Congress of Mathematicians, as a 
challenge to mathematicians at the birth of the twentieth century. Since that time, they have spurred a tremendous 
amount and variety of research. Although many of these problems have now been solved, several remain open, 


including the Riemann hypothesis, which is part of Problem 8 on Hilbert’s list. Hilbert was also the author of several important 
textbooks in number theory and geometry. 
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EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


Take room 1, 
everyone else 
move down one room 


f 


Manager New guest 


FIGURE 2 A New Guest Arrives at Hilbert’s Grand Hotel. 


How can we accommodate a new guest arriving at the fully occupied Grand Hotel without 
removing any of the current guests? 


Solution: Because the rooms of the Grand Hotel are countable, we can list them as Room 1, 
Room 2, Room 3, and so on. When a new guest arrives, we move the guest in Room | to Room 
2, the guest in Room 2 to Room 3, and in general, the guest in Room n to Room n + 1, for all 
positive integers n. This frees up Room 1, which we assign to the new guest, and all the current 
guests still have rooms. We illustrate this situation in Figure 2. 


When there are finitely many room in a hotel, the notion that all rooms are occupied is 
equivalent to the notion that no new guests can be accommodated. However, Hilbert’s paradox 
of the Grand Hotel can be explained by noting that this equivalence no longer holds when there 
are infinitely many room. 


EXAMPLES OF COUNTABLE AND UNCOUNTABLE SETS We will now show that cer- 
tain sets of numbers are countable. We begin with the set of all integers. Note that we can show 
that the set of all integers is countable by listing its members. 


Show that the set of all integers is countable. 


Solution: We can list all integers in a sequence by starting with O and alternating between 
positive and negative integers: 0, 1, —1,2, —2,.... Alternatively, we could find a one-to-one 
correspondence between the set of positive integers and the set of all integers. We leave it to the 
reader to show that the function f(n) = n/2 when n is even and f(n) = —(n — 1)/2 when n 
is odd is such a function. Consequently, the set of all integers is countable. 


It is not surprising that the set of odd integers and the set of all integers are both countable 
sets (as shown in Examples | and 3). Many people are amazed to learn that the set of rational 
numbers is countable, as Example 4 demonstrates. 


Show that the set of positive rational numbers is countable. 


Solution: It may seem surprising that the set of positive rational numbers is countable, but we 
will show how we can list the positive rational numbers as a sequencer], 72,...,/n,..-. First, 
note that every positive rational number is the quotient p/q of two positive integers. We can 
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Terms not circled C 
are not listed 


because they 


repeat previously 3 
listed terms VA, 3 


2 3 4 B} 
i. 4 4 4 

2 4 

5 


WMI 
| 
oe UN 


FIGURE 3 The Positive Rational Numbers Are Countable. 


arrange the positive rational numbers by listing those with denominator q = | in the first row, 
those with denominator g = 2 in the second row, and so on, as displayed in Figure 3. 

The key to listing the rational numbers in a sequence is to first list the positive rational 
numbers p/q with p+ q = 2, followed by those with p+ q = 3, followed by those with 
p+4q =4,and so on, following the path shown in Figure 3. Whenever we encounter a number 
P/q that is already listed, we do not list it again. For example, when we come to 2/2 = | we 
do not list it because we have already listed 1/1 = 1. The initial terms in the list of positive 
rational numbers we have constructed are 1, 1/2, 2, 3, 1/3, 1/4, 2/3, 3/2, 4, 5, and so on. These 
numbers are shown circled; the uncircled numbers in the list are those we leave out because 
they are already listed. Because all positive rational numbers are listed once, as the reader can 
verify, we have shown that the set of positive rational numbers is countable. < 


An Uncountable Set 


Not all infinite sets have 


‘ocamecios! We have seen that the set of positive rational numbers is a countable set. Do we have a promising 


candidate for an uncountable set? The first place we might look is the set of real numbers. In 

Example 5 we use an important proof method, introduced in 1879 by Georg Cantor and known 
links Qk’ as the Cantor diagonalization argument, to prove that the set of real numbers is not countable. 
This proof method is used extensively in mathematical logic and in the theory of computation. 


EXAMPLE 5 — Show that the set of real numbers is an uncountable set. 


Extra a Solution: To show that the set of real numbers is uncountable, we suppose that the set of real 
Examples Q& numbers is countable and arrive at a contradiction. Then, the subset of all real numbers that 
fall between 0 and | would also be countable (because any subset of a countable set is also 

countable; see Exercise 16). Under this assumption, the real numbers between 0 and | can be 

listed in some order, say, 11, 72, 73, .. .. Let the decimal representation of these real numbers be 


r1 = 0.d\1dj2d13d 14... 
ro = 0.d21d22d23d24 ... 
30> 0.d31d32d33d34 sere 
ra = 0.d4,d4od43d4q ... 


where dj; € {0, 1, 2, 3, 4,5, 6, 7, 8, 9}. (For example, if r) = 0.23794102..., we have d); = 
2, dj2 = 3, dj3 =7, and so on.) Then, form a new real number with decimal expansion 
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? 


A number with a decimal 
expansion that terminates 
has a second decimal 
expansion ending with an 
infinite sequence of 9s 
because 1 = 0.999... 


THEOREM 1 


This proof uses WLOG 
and cases. 


THEOREM 2 


r = 0.d\dod3d4..., where the decimal digits are determined by the following rule: 


_ fifi #4 
' 15 ifdj; = 4. 


(As an example, suppose that r; = 0.23794102..., rg =0.44590138..., m= 
0.09118764..., rq = 0.80553900..., and so on. Then we have r = 0.d,d2d3d4...= 
0.4544 ..., where dj = 4 because d1; 4 4, dz = 5 because d22 = 4, d3 = 4 because d33 £ 4, 
d4 = 4 because d44 4 4, and so on.) 

Every real number has a unique decimal expansion (when the possibility that the expansion 
has a tail end that consists entirely of the digit 9 is excluded). Therefore, the real number r is not 
equal to any of 71, r2, . .. because the decimal expansion of r differs from the decimal expansion 
of r; in the ith place to the right of the decimal point, for each 7. 

Because there is areal number r between 0 and | that is not in the list, the assumption that all 
the real numbers between 0 and | could be listed must be false. Therefore, all the real numbers 
between 0 and | cannot be listed, so the set of real numbers between 0 and | is uncountable. Any 
set with an uncountable subset is uncountable (see Exercise 15). Hence, the set of real numbers 
is uncountable. 


RESULTS ABOUT CARDINALITY We will now discuss some results about the cardinality 
of sets. First, we will prove that the union of two countable sets is also countable. 


If A and B are countable sets, then A U B is also countable. 


Proof: Suppose that A and B are both countable sets. Without loss of generality, we can assume 
that A and B are disjoint. (If they are not, we can replace B by B — A, because AN (B — A) = 9 
and AU (B— A) = AUB.) Furthermore, without loss of generality, if one of the two sets is 
countably infinite and other finite, we can assume that B is the one that is finite. 

There are three cases to consider: (i) A and B are both finite, (ii) A is infinite and B is finite, 
and (i11) A and B are both countably infinite. 


Case (i): Note that when A and B are finite, A U B is also finite, and therefore, countable. 


Case (ii): Because A is countably infinite, its elements can be listed in an infinite sequence 


a1, 42, 43, ..., An, ... and because B is finite, its terms can be listed as bj, bo, ..., bm for 

some positive integer m. We can list the elements of A U B as bj, bo, ..., bm, a1, 42, 43, 
.., Gn, --.. This means that A U B is countably infinite. 

Case (iii): Because both A and B are countably infinite, we can list their elements as aj, 

a2, 43, ..-, An, ... and by, bo, b3, ..., bn, ..., respectively. By alternating terms of these 

two sequences we can list the elements of A U B in the infinite sequence a, bj, a, b2, a3, 

b3, ..., An, bn, .... This means A U B must be countably infinite. 


We have completed the proof, as we have shown that A U B is countable in all three 
cases. q 


Because of its importance, we now state a key theorem in the study of cardinality. 


SCHRODER-BERNSTEIN THEOREM If A and B are sets with |A| < |B| and |B| < 
|A|, then |A| = |B|. In other words, if there are one-to-one functions f from A to B and g 
from B to A, then there is a one-to-one correspondence between A and B. 


EXAMPLE 6 


DEFINITION 4 


c is the lowercase 
Fraktur c. 
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Because Theorem 2 seems to be quite straightforward, we might expect that it has an easy 
proof. However, even though it can be proved without using advanced mathematics, no known 
proof is easy to explain. Consequently, we omit a proof here. We refer the interested reader to 
[AiZiHo09] and [Ve06] for a proof. This result is called the Schréder-Bernstein theorem after 
Ernst Schréder who published a flawed proof of it in 1898 and Felix Bernstein, a student of 
Georg Cantor, who presented a proof in 1897. However, a proof of this theorem was found 
in notes of Richard Dedekind dated 1887. Dedekind was a German mathematician who made 
important contributions to the foundations of mathematics, abstract algebra, and number theory. 
We illustrate the use of Theorem 2 with an example. 


Show that the |(O, 1)| = |(0, L]]. 


Solution: It is not at all obvious how to find a one-to-one correspondence between (0, 1) and 
(0, 1] to show that |(O, 1)| = |(0, 1]|. Fortunately, we can use the Schréder-Bernstein theorem 
instead. Finding a one-to-one function from (0, 1) to (0, 1] is simple. Because (0, 1) Cc (0, 1], 
f(x) =x is a one-to-one function from (0, 1) to (0, 1]. Finding a one-to-one function from 
(0, 1] to (0, 1) is also not difficult. The function g(x) = x/2 is clearly one-to-one and maps 
(0, 1] to (0, 1/2] Cc (0, 1). As we have found one-to-one functions from (0, 1) to (0, 1] and 
from (0, 1] to (O, 1), the Schréder-Bernstein theorem tells us that |(O, 1)| = |(0, 1]]. < 


UNCOMPUTABLE FUNCTIONS We will now describe an important application of the 
concepts of this section to computer science. In particular, we will show that there are functions 
whose values cannot be computed by any computer program. 


We say that a function is computable if there is a computer program in some programming 
language that finds the values of this function. If a function is not computable we say it is 
uncomputable. 


To show that there are uncomputable functions, we need to establish two results. First, we 
need to show that the set of all computer programs in any particular programming language is 
countable. This can be proved by noting that a computer programs in a particular language can 
be thought of as a string of characters from a finite alphabet (see Exercise 37). Next, we show 
that there are uncountably many different functions from a particular countably infinite set to 
itself. In particular, Exercise 38 shows that the set of functions from the set of positive integers 
to itself is uncountable. This is a consequence of the uncountability of the real numbers between 
0 and | (see Example 5). Putting these two results together (Exercise 39) shows that there are 
uncomputable functions. 


THE CONTINUUM HYPOTHESIS We conclude this section with a brief discussion of a 
famous open question about cardinality. It can be shown that the power set of Z* and the set 
of real numbers R have the same cardinality (see Exercise 38). In other words, we know that 
|P(Z*)| = |R| = c, where c denotes the cardinality of the set of real numbers. 

Animportant theorem of Cantor (Exercise 40) states that the cardinality of a set is always less 
than the cardinality of its power set. Hence, |Z*| < |P(Z*)|. We can rewrite this as No < 2%, 
using the notation 2!5! to denote the cardinality of the power set of the set S. Also, note that the 
relationship |P(Z+)| = |R| can be expressed as 2° = ¢. 

This leads us to the famous continuum hypothesis, which asserts that there is no cardinal 
number X between Xo and c. In other words, the continuum hypothesis states that there is no set 
A such that Xo, the cardinality of the set of positive integers, is less than |A| and |A| is less than 
c, the cardinality of the set of real numbers. It can be shown that the smallest infinite cardinal 
numbers form an infinite sequence Xo < 8; < 2 <--- . If we assume that the continuum 
hypothesis is true, it would follow that c = &1, so that 2%0 =). 
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The continuum hypothesis was stated by Cantor in 1877. He labored unsuccessfully to prove 
it, becoming extremely dismayed that he could not. By 1900, settling the continuum hypothesis 
was considered to be among the most important unsolved problems in mathematics. It was the 
first problem posed by David Hilbert in his famous 1900 list of open problems in mathematics. 

The continuum hypothesis is still an open question and remains an area for active research. 
However, it has been shown that it can be neither proved nor disproved under the standard set 
theory axioms in modern mathematics, the Zermelo-Fraenkel axioms. The Zermelo-Fraenkel 
axioms were formulated to avoid the paradoxes of naive set theory, such as Russell’s paradox, 
but there is much controversy whether they should be replaced by some other set of axioms for 


set theory. 


Exercises 


1. Determine whether each of these sets is finite, countably 


infinite, or uncountable. For those that are countably in- 

finite, exhibit a one-to-one correspondence between the 

set of positive integers and that set. 

a) the negative integers 

b) the even integers 

c) the integers less than 100 

d) the real numbers between 0 and 5 

e) the positive integers less than 1,000,000,000 

f) the integers that are multiples of 7 

. Determine whether each of these sets is finite, countably 

infinite, or uncountable. For those that are countably in- 

finite, exhibit a one-to-one correspondence between the 

set of positive integers and that set. 

a) the integers greater than 10 

b) the odd negative integers 

c) the integers with absolute value less than 1,000,000 

d) the real numbers between 0 and 2 

e) the set A x Zt where A = {2, 3} 

f) the integers that are multiples of 10 

. Determine whether each of these sets is countable or un- 

countable. For those that are countably infinite, exhibit 

a one-to-one correspondence between the set of positive 

integers and that set. 

a) all bit strings not containing the bit 0 

b) all positive rational numbers that cannot be written 
with denominators less than 4 

c) the real numbers not containing 0 in their decimal 
representation 

d) the real numbers containing only a finite number of 
1s in their decimal representation 

. Determine whether each of these sets is countable or un- 

countable. For those that are countably infinite, exhibit 

a one-to-one correspondence between the set of positive 

integers and that set. 

a) integers not divisible by 3 

b) integers divisible by 5 but not by 7 

c) the real numbers with decimal representations con- 
sisting of all Is 

d) the real numbers with decimal representations of all 
Is or 9s 


*9, 


10. 


11. 


. Show that a finite group of guests arriving at Hilbert’s 


fully occupied Grand Hotel can be given rooms without 
evicting any current guest. 


. Suppose that Hilbert’s Grand Hotel is fully occupied, but 


the hotel closes all the even numbered rooms for mainte- 
nance. Show that all guests can remain in the hotel. 


. Suppose that Hilbert’s Grand Hotel is fully occupied on 


the day the hotel expands to a second building which also 
contains a countably infinite number of rooms. Show that 
the current guests can be spread out to fill every room of 
the two buildings of the hotel. 


. Show that a countably infinite number of guests arriv- 


ing at Hilbert’s fully occupied Grand Hotel can be given 
rooms without evicting any current guest. 

Suppose that a countably infinite number of buses, each 
containing a countably infinite number of guests, arrive 
at Hilbert’s fully occupied Grand Hotel. Show that all the 
arriving guests can be accommodated without evicting 
any current guest. 

Give an example of two uncountable sets A and B such 
that A — B is 

a) finite. 

b) countably infinite. 

c) uncountable. 

Give an example of two uncountable sets A and B such 
that A/ B is 

a) finite. 

b) countably infinite. 

c) uncountable. 


. Show that if A and B are sets and A C B then |A| < |B]. 
. Explain why the set A is countable if and only if |A| < 


|Z*|. 


. Show that if A and B are sets with the same cardinality, 


then |A| <|B| and |B| < |A|. 


. Show that if A and B are sets, A is uncountable, and 


A C B, then B is uncountable. 


. Show that a subset of a countable set is also countable. 


. If A is an uncountable set and B is a countable set, must 


A — B be uncountable? 


18. 


19. 


20. 
21. 


22. 


23. 


24. 


25. 


26. 


*27, 


28. 
*29, 
* 30. 


*31. 


* 32. 


Show that if A and B are sets |A| = |B], then |P(A)| = 
|P(B)|. 

Show that if A, B, C, and D are sets with |A| = |B| and 
|C| = |D|, then |A x C| = |B x DI. 

Show that if |A| = |B| and |B| = |C|, then |A| = |C|. 
Show that if A, B, and C are sets such that |A| < |B| and 
|B| <|C|, then |A] < |C]. 

Suppose that A is a countable set. Show that the set B is 
also countable if there is an onto function f from A to B. 


Show that if A is an infinite set, then it contains a count- 
ably infinite subset. 


Show that there is no infinite set A such that |A| < |Z*| = 
Xo. 

Prove that if it is possible to label each element of an 
infinite set S with a finite string of keyboard characters, 
from a finite list characters, where no two elements of S$ 
have the same label, then S is a countably infinite set. 


Use Exercise 25 to provide a proof different from that 
in the text that the set of rational numbers is countable. 
[Hint: Show that you can express a rational number as a 
string of digits with a slash and possibly a minus sign.] 


Show that the union of a countable number of countable 
sets is countable. 


Show that the set Zt x Z* is countable. 
Show that the set of all finite bit strings is countable. 


Show that the set of real numbers that are solutions of 
quadratic equations ax-+bx+c= 0, where a, b, andc 
are integers, is countable. 


Show that Zt x Zt is countable by showing that 
the polynomial function f:Z* x Zt — Zt with 
f(m,n) = (m+n—2)(m+n—1)/2+m 1s one-to- 
one and onto. 

Show that when you substitute (3n + 1)? for each occur- 
rence of n and (3m + 1)? for each occurrence of m in the 
right-hand side of the formula for the function f(m, 7) 
in Exercise 31, you obtain a one-to-one polynomial func- 
tion Z x Z — Z. It is an open question whether there is 
a one-to-one polynomial function Q x Q—> Q. 


Matrices 


33. 


34. 


35. 


* 36. 


* 37. 


* 38. 


* 39, 


* 40. 
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Use the Schréder-Bernstein theorem to show that (0, 1) 
and [0, 1] have the same cardinality 


Show that (0, 1) and R have the same cardinality. [Hint: 
Use the Schréder-Bernstein theorem. ] 


Show that there is no one-to-one correspondence from 
the set of positive integers to the power set of the set of 
positive integers. [Hint: Assume that there is such a one- 
to-one correspondence. Represent a subset of the set of 
positive integers as an infinite bit string with ith bit 1 if7 
belongs to the subset and 0 otherwise. Suppose that you 
can list these infinite strings in a sequence indexed by the 
positive integers. Construct a new bit string with its ith 
bit equal to the complement of the ith bit of the ith string 
in the list. Show that this new bit string cannot appear in 
the list.] 

Show that there is a one-to-one correspondence from the 
set of subsets of the positive integers to the set real num- 
bers between 0 and 1. Use this result and Exercises 34 and 
35 to conclude that Xo < |P(Z*)| = |R|. [Hint: Look at 
the first part of the hint for Exercise 35.] 


Show that the set of all computer programs in a partic- 
ular programming language is countable. [Hint: A com- 
puter program written in a programming language can be 
thought of as a string of symbols from a finite alphabet.] 
Show that the set of functions from the positive inte- 
gers to the set {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} is uncountable. 
[Hint: First set up a one-to-one correspondence between 
the set of real numbers between 0 and 1 and a subset of 
these functions. Do this by associating to the real number 
0.djdz2...dy,... the function f with f(n) = dy.] 

We say that a function is computable if there is a com- 
puter program that finds the values of this function. Use 
Exercises 37 and 38 to show that there are functions that 
are not computable. 


Show that if S is a set, then there does not exist an onto 
function f from S$ to P(S), the power set of S. Con- 
clude that |$| < |P(S)|. This result is known as Cantor’s 
theorem. [Hint: Suppose such a function f existed. Let 
T = {s € S| s ¢ f(s)} and show that no element s can 
exist for which f(s) = T.] 


Introduction 


Matrices are used throughout discrete mathematics to express relationships between elements 
in sets. In subsequent chapters we will use matrices in a wide variety of models. For instance, 
matrices will be used in models of communications networks and transportation systems. Many 
algorithms will be developed that use these matrix models. This section reviews matrix arithmetic 


that will be used in these algorithms. 
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DEFINITION 1 — A matrix is a rectangular array of numbers. A matrix with m rows and n columns is called 
an m x n matrix. The plural of matrix is matrices. A matrix with the same number of rows 
as columns is called square. Two matrices are equal if they have the same number of rows 
and the same number of columns and the corresponding entries in every position are equal. 


1 1 
EXAMPLE1 Thematrix |0 2] isa3 x 2 matrix. < 
1 3 


We now introduce some terminology about matrices. Boldface uppercase letters will be 
used to represent matrices. 


DEFINITION 2 — Let m and n be positive integers and let 


a1 a2 tee Qn 
a21 a22 One & a2n 


es | 
ee Am2_ --- mel 
The ith row of A is the | x n matrix [a;1, aj2,..., Gin]. The jth column of A is the m x 1 
matrix 
aj 


al 


The (i, j)th element or entry of A is the element a;;, that is, the number in the ith row and 
jth column of A. A convenient shorthand notation for expressing the matrix A is to write 
A = [a,j], which indicates that A is the matrix with its (i, 7)th element equal to aj. 


Matrix Arithmetic 


The basic operations of matrix arithmetic will now be discussed, beginning with a definition of 
matrix addition. 


DEFINITION 3 Let A = [aj] and B = [b,] be m x n matrices. The sum of A and B, denoted by A + B, is 
the m x n matrix that has a; + bj as its (i, j)th element. In other words, A + B = [a; + 5]. 


The sum of two matrices of the same size is obtained by adding elements in the corresponding 
positions. Matrices of different sizes cannot be added, because the sum of two matrices is defined 
only when both matrices have the same number of rows and the same number of columns. 


EXAMPLE 2 
1 O -l 3 4 -1 4 4 -2 
Wehave}2 2 —3]+ 1 -3 O;/=)3 -1 -3). < 
3 4 0 —1 1 2 2 5 2 


DEFINITION 4 


EXAMPLE 3 


Extra p> 
Examples 
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We now discuss matrix products. A product of two matrices is defined only when the number 
of columns in the first matrix equals the number of rows of the second matrix. 


Let A be anm x k matrix and B beak x n matrix. The product of A and B, denoted by AB, is 
the m x n matrix with its (7, j)th entry equal to the sum of the products of the corresponding 
elements from the ith row of A and the jth column of B. In other words, if AB = [c;;], then 


Cy = aj1b1j + aj2boj + +++ + ajxdyj- 


In Figure | the colored row of A and the colored column of B are used to compute the element 
cjj of AB. The product of two matrices is not defined when the number of columns in the first 
matrix and the number of rows in the second matrix are not the same. 

We now give some examples of matrix products. 


Let 
44 2 4 
A= and B=)]1 1 
3 1 0 3 0 
0 2 2 
Find AB if it is defined. 


Solution: Because A is a4 x 3 matrix and B is a3 x 2 matrix, the product AB is defined and is 
a4 x 2 matrix. To find the elements of AB, the corresponding elements of the rows of A and the 
columns of B are first multiplied and then these products are added. For instance, the element in 
the (3, 1)th position of AB is the sum of the products of the corresponding elements of the third 
row of A and the first column of B; namely, 3-2 + 1-1-+0-3 = 7. When all the elements of 
AB are computed, we see that 


it 4 
aB=| 2 91. 
7 23 < 
g 2 


Matrix multiplication is not commutative. That is, if A and B are two matrices, it is not 
necessarily true that AB and BA are the same. In fact, it may be that only one of these two 
products is defined. For instance, if A is 2 x 3 and B is 3 x 4, then AB is defined and is 2 x 4; 
however, BA is not defined, because it is impossible to multiply a 3 x 4 matrix and a 2 x 3 
matrix. 

In general, suppose that A is an m x n matrix and B is anr x s matrix. Then AB is defined 
only when n = r and BA is defined only when s = m. Moreover, even when AB and BA are 


ay Q12. «++ Aik 
E a22——« ++ = ADK by by... Daj Din Cll C12 Cin 
| : : : | & bo... b>; | E C29. 04 | 
he a wal - 4 : aan : ig a 
be bea we Dj bkn Cm1 Cm2 --- Cmn 


Pe Am2 ++. Fa 


FIGURE 1 The Product of A = [a;;] and B= [);;]. 
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EXAMPLE 4 


DEFINITION 5 


both defined, they will not be the same size unless m = n = r = s. Hence, if both AB and BA 
are defined and are the same size, then both A and B must be square and of the same size. 
Furthermore, even with A and B bothn x n matrices, AB and BA are not necessarily equal, as 
Example 4 demonstrates. 


Let 


Does AB = BA? 


Solution: We find that 


ap =|; | aid Ba =|3 al 


Hence, AB # BA. < 


Transposes and Powers of Matrices 


We now introduce an important matrix with entries that are zeros and ones. 


The identity matrix of order n is the n x n matrix I, = [6;;], where 6;; = 1 if i = j and 
dj; = Oifi A j. Hence 


TO cree 0) 
@ 


eae| 


Multiplying a matrix by an appropriately sized identity matrix does not change this matrix. In 
other words, when A is an m xX n matrix, we have 


AL, =InA =A. 


Powers of square matrices can be defined. When A is ann x n matrix, we have 


The operation of interchanging the rows and columns of a square matrix arises in many 
contexts. 
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DEFINITION 6 Let A = [a;;] be an m x n matrix. The transpose of A, denoted by A’, is the n x m matrix 
obtained by interchanging the rows and columns of A. In other words, if A‘ = [b;;], then 
hy igi Wore = Ml, soon tt CuCl fj = Ih, oo ante 


1 4 


EXAMPLE 5 The transpose of the matrix Le is the matrix |2 5]. 
4 5 6 3 6 < 
Matrices that do not change when their rows and columns are interchanged are often im- 


portant. 


DEFINITION 7 _ A square matrix A is called symmetric if A = A‘. Thus A = [a;;] is symmetric if aj; = aji 
for alli and j with 1 <i <nand1l <j <n. 


Note that a matrix is symmetric if and only if it is square and it is symmetric with respect to its 
main diagonal (which consists of entries that are in the ith row and ith column for some 7). This 
symmetry is displayed in Figure 2. 


0 


1 1 
EXAMPLE6 Thematrix}1 0 is symmetric. 4 
0 1 


1 
0 


Zero—One Matrices 


A matrix all of whose entries are either 0 or | is called a zero—one matrix. Zero—one matrices 
FIGURE2 A are often used to represent discrete structures, as we will see in Chapters 9 and 10. Algorithms 
Symmetric Matrix. using these structures are based on Boolean arithmetic with zero—one matrices. This arithmetic 
is based on the Boolean operations A and v, which operate on pairs of bits, defined by 


1 ifbb=b=1 
bh Abng= . 
0 otherwise, 
iahie 1 iff} =lormy=1 
: — 0 otherwise. 


DEFINITION 8 _ Let A = [a;;] and B = [b;;] be m x n zero—one matrices. Then the join of A and B is the 
zero—one matrix with (i, j)th entry a;; V b;;. The join of A and B is denoted by A v B. The 
meet of A and B is the zero—one matrix with (i, j)th entry aj; A bjj. The meet of A and B is 
denoted by AA B. 


EXAMPLE 7 _ Find the join and meet of the zero—one matrices 


10.1 0 1 0 
a=[o 1 a B=|i 1 a; 
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Solution: We find that the join of A and B is 
1v0O Ovl tIv0O 1 1 1 
ave=[5y4 lvl ar eae 1 a 
The meet of A and B is 
1A0 OAL LAO 0 0 O 
anB=|o ni LAl onok= le 1 Ok < 


We now define the Boolean product of two matrices. 


DEFINITION 9 — LetA = [a;;] be anm x k zero—one matrix and B = [b;;] be ak x n zero—one matrix. Then 
the Boolean product of A and B, denoted by A©B, is the m x n matrix with (7, 7)th entry 
cj where 


Cij = (ait A bij) V (diz A baj) V ++ V (Gik A xj). 


Note that the Boolean product of A and B is obtained in an analogous way to the ordinary 
product of these matrices, but with addition replaced with the operation Vv and with multiplication 
replaced with the operation A. We give an example of the Boolean products of matrices. 


EXAMPLE 8 | Find the Boolean product of A and B, where 


Solution: The Boolean product A © B is given by 


QGAAI)V@OAN) GALIDNVOAIL CGAADVOA1) 
AOB=]OAIVQUA0) OAIVAAIL OADVAAIL 
GAIVO@OAN) GALDNVOAIL GAADVOAI) 


1v0O Iv0O OvO 
=!10v0 Ovi Ovi 
1v0O Iv0O Ovo 


1 1 0 
=/0 1 1]. 
1 1 0 < 


We can also define the Boolean powers of a square zero—one matrix. These powers will 
be used in our subsequent studies of paths in graphs, which are used to model such things as 
communications paths in computer networks. 
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DEFINITION 10 — Let A bea square zero—one matrix and let r be a positive integer. The rth Boolean power of 
A is the Boolean product of r factors of A. The rth Boolean product of A is denoted by Al”!, 
Hence 


A"l= AOAOAO..-. OA. 
—— 


r times 


(This is well defined because the Boolean product of matrices is associative.) We also define 
All to be In. 


0 0 1 
EXAMPLE9 LetA=]|1 0 0]. Find Al”! for all positive integers n. 
1 1 O 


Solution: We find that 


1 1 O 
APIZAOA=]0 0 1 
1 0 1 
We also find that 
1 0 1 1 1 1 
APM AMOA=]1 1 0], AM=AaPloa=]1 0 1 
1 1 1 1 1 1 
Additional computation shows that 
1 1 1 
ADl=}]1 1 
1 1 1 
The reader can now see that A!”! = Al! for all positive integers n with n > 5. < 
Exercises 
1113 (= Se < 
tia | 0 4 “| bA=[74 =3 5 3i; 
1 1 3 =7 
a) What size is A? B= ra = a 5 
b) What is the third column of A? . - 
c) What is the second row of A? 3. Find AB if 
d) What is the element of A in the (3, 2)th position? aH 2 1 B= 0 4 
e) What is A’? ~13 2)?" —J1 3y° 
2. Find A + B, where 1 -l 3 2 1 
1 0 4 b)A=]0 1 Bel 0 ei 
VAL B. By a. -4 
O- <2: 3 3 
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4. 


10. 


11. 


12. 


13. 


14. 


Find the product AB, where 
1 0 1 0 1 -l 
a) A= 0 1 1)},B= 1 1 0 
-l1 1 0 -1 0 1 
1 -3 0 1 -1 2 3 
b) A=] 1 2 21,B=|-1 0 3 -i1 
2 1 -l —3 -—2 0 2 
0 -i1 
-4 —3 


. Find a matrix A such that 


2 3 3 0 
ft aJ+=[1 
[Hint: Finding A requires that you solve systems of linear 
equations. ] 


. Find a matrix A such that 


ie 
21 
4 0 


. Let A be anm x n matrix and let 0 be the m x n matrix 


that has all entries equal to zero. Show thatA = 0+ A = 
A+0. 


. Show that matrix addition is commutative; that is, 


show that if A and B are both m xn matrices, then 
A+B=B-+4A. 


. Show that matrix addition is associative; that is, show 


that if A, B, and C are all m xn matrices, then 
A+ (B+ C) = (A+B)+C. 


Let A be a3 x 4 matrix, B be a4 x 5 matrix, and C bea 
4 x 4 matrix. Determine which of the following products 
are defined and find the size of those that are defined. 


a) AB b) BA c) AC 
d) CA e) BC f) CB 
What do we know about the sizes of the matrices A and 


B if both of the products AB and BA are defined? 


In this exercise we show that matrix multiplication is dis- 
tributive over matrix addition. 


a) Suppose that A and B are m x k matrices and that C 
isak x n matrix. Show that (A + B)C = AC + BC. 


b) Suppose that Cis anm x k matrix and that A and B are 

k x n matrices. Show that C(A + B) = CA + CB. 
In this exercise we show that matrix multiplication is 
associative. Suppose that A is an m x p matrix, B is 
a p xk matrix, and C is a k xn matrix. Show that 
A(BC) = (AB)C. 


Then x n matrix A = [q;;] is called a diagonal matrix if 
aij =O wheni ¥ j. Show that the product of twon x n 
diagonal matrices is again a diagonal matrix. Give a sim- 
ple rule for determining this product. 


15. 


16. 
17. 


Let 


Find a formula for A”, whenever n is a positive integer. 
Show that (A‘)! = A. 

Let A and B be twon x n matrices. Show that 

a) (A+B)' =A’ +B". 

b) (AB)! = BYA’. 


If A and B are n x n matrices with AB = BA = I, then B 
is called the inverse of A (this terminology is appropriate be- 
cause such a matrix B is unique) and A is said to be invertible. 
The notation B = A~! denotes that B is the inverse of A. 


18. 


19, 


20. 


21. 


22. 


23. 


Show that 
2 3 -1 
1 2 1 
-1 -l 3 


is the inverse of 


Tee, 5 
ad. 35° a 
1 =. 4 


Let A be the 2 x 2 matrix 


a b 
ve |< | . 
Show that if ad — bc £ 0, then 
d —b 
ad—bc ad—bce 


=, a 


ad—bc ad—bce 


AT! = 


Let 


a) Find A~!. [Hint: Use Exercise 19.] 

b) Find A?. 

c) Find (A7!)3. 

d) Use your answers to (b) and (c) to show that (A7!)3 
is the inverse of A?. 

Let A be an invertible matrix. Show that (A”)~! = 

(A~!)” whenever n is a positive integer. 

Let A be a matrix. Show that the matrix AA‘ is symmet- 

ric. [Hint: Show that this matrix equals its transpose with 

the help of Exercise 17b.] 

Suppose that A is an n x n matrix where n is a positive 

integer. Show that A + A‘ is symmetric. 


24. a) Show that the system of simultaneous linear equations 


ayix1 +aj2x2 +-:- 


a2\X1 + 22x2 +-°- 


AniX1 + an2xX2 ++: 


in the variables x,, x2,.. 


+ GinXn = by 


+ AnXn = b2 


+ AnnXn = Dp. 


., Xp can be expressed as 


AX = B, where A = [a;;], X is ann x | matrix with 


x; the entry in its ith row, and B is ann x | matrix 
with b; the entry in its ith row. 


b) Show that if the matrix A = [a;;] is invertible (as 
defined in the preamble to Exercise 18), then the so- 
lution of the system in part (a) can be found using the 
equation X = A7!B. 


25. Use Exercises 18 and 24 to solve the system 


7x, — 8x2 +52x3 =5 
—4x; + 5x2 — 3x3 = -3 


xj — x2 +x3=0 


26. Let 
1 1 0 1 
a=| | and B= |} | 
Find 
a) AVB. b) AAB. c) ACB. 
27. Let 
1041 0141 
A=]1 1 0 and B=]1 0 1 
0 0 1 101 
Find 
a) AVB. b) AAB. c) ACB. 


Key Terms and Results 


TERMS 

set: a collection of distinct objects 

axiom: a basic assumption of a theory 

paradox: a logical inconsistency 

element, member of a set: an object in a set 

roster method: a method that describes a set by listing its 
elements 

set builder notation: the notation that describes a set by stating 
a property an element must have to be a member 

% (empty set, null set): the set with no members 

universal set: the set containing all objects under considera- 
tion 

Venn diagram: a graphical representation of a set or sets 

S = T (set equality): S and T have the same elements 
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28. Find the Boolean product of A and B, where 


1001 P : 
A=]0 1 0 1 and B= 
1111 i 
1 0 
29. Let 
1 0 
A=]1 0 1 
0 1 
Find 
a) Al], b) AGI, 


c) AVAPI y ABI, 

30. Let A be a zero—one matrix. Show that 
a) AVA=A. b) AAA=A. 

31. In this exercise we show that the meet and join opera- 
tions are commutative. Let A and B be m x n zero—one 
matrices. Show that 
a) AVB=BVA. b) BAA=AAB. 

32. In this exercise we show that the meet and join opera- 
tions are associative. Let A, B, and C be m x n zero—one 
matrices. Show that 
a) (AVB)VC=AVv (BVOC). 

b) AAB)AC=AA (BAC). 

33. We will establish distributive laws of the meet over the 
join operation in this exercise. Let A, B, and C bem x n 
zero—one matrices. Show that 
a) AV (BAC) =(AVB)A (AVC). 

b) AA (BV ©) = (AAB)V (AAC). 

34. Let A be ann x n zero—one matrix. Let I be then xn 
identity matrix. Show that AOT=IOA=A. 

35. In this exercise we will show that the Boolean prod- 
uct of zero—one matrices is associative. Assume that A 
is an m X p zero—one matrix, B is a p x k zero—one 


matrix, and C is a k x n zero—one matrix. Show that 
AO (BOC) = (AOB) OC. 


S CT (S is a subset of T): every element of S is also an 
element of T 


S c T (S is a proper subset of 7): S is a subset of T and 
SAT 

finite set: a set with n elements, where n is a nonnegative 
integer 

infinite set: a set that is not finite 

|S| (the cardinality of S): the number of elements in S 

P(S) (the power set of S): the set of all subsets of S$ 

A U B (the union of A and B): the set containing those ele- 
ments that are in at least one of A and B 

AN B (the intersection of A and B): the set containing those 
elements that are in both A and B. 
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A — B (the difference of A and B): the set containing those 
elements that are in A but not in B 

A (the complement of A): the set of elements in the universal 
set that are not in A 

A @® B (the symmetric difference of A and B): the set con- 
taining those elements in exactly one of A and B 

membership table: a table displaying the membership of ele- 
ments in sets 

function from A to B: an assignment of exactly one element 
of B to each element of A 

domain of f: the set A, where f is a function from A to B 

codomain of f: the set B, where f is a function from A to B 

b is the image of a under f: b = f(a) 

ais a pre-image of b under f: f(a) = b 

range of f: the set of images of f 

onto function, surjection: a function from A to B such that 
every element of B is the image of some element in A 

one-to-one function, injection: a function such that the im- 
ages of elements in its domain are distinct 

one-to-one correspondence, bijection: a function that is both 
one-to-one and onto 

inverse of f: the function that reverses the correspondence 
given by f (when f is a bijection) 

f ° g (composition of f and g): the function that assigns 
f(g(x)) tox 

|x] (floor function): the largest integer not exceeding x 

[x] (ceiling function): the smallest integer greater than or 
equal to x 

partial function: an assignment to each element in a subset of 
the domain a unique element in the codomain 

sequence: a function with domain that is a subset of the set of 
integers 

geometric progression: a sequence of the forma, ar, ar?, baad 
where a and r are real numbers 

arithmetic progression: a sequence of the form a, a+d, 
a-+2d,..., where a and d are real numbers 

string: a finite sequence 

empty string: a string of length zero 

recurrence relation: a equation that expresses the nth term a, 
of a sequence in terms of one or more of the previous terms 
of the sequence for all integers n greater than a particular 
integer 


Review Questions 


1. Explain what it means for one set to be a subset of another 
set. How do you prove that one set is a subset of another 
set? 

2. What is the empty set? Show that the empty set is a subset 
of every set. 

3. a) Define | S|, the cardinality of the set S. 

b) Give a formula for |A U B|, where A and B are sets. 

4. a) Define the power set of a set S. 

b) When is the empty set in the power set of a set S? 


c) How many elements does the power set of a set S with 
n elements have? 


eye ait the sum ay + ay +--+ + ay 

Th} a1 i: the product ajaz --+ ay 

cardinality: two sets A and B have the same cardinality if 
there is a one-to-one correspondence from A to B 

countable set: a set that either is finite or can be placed in 
one-to-one correspondence with the set of positive integers 


uncountable set: a set that is not countable 

Xo (aleph null): the cardinality of a countable set 

c: the cardinality of the set of real numbers 

Cantor diagonalization argument: a proof technique used to 
show that the set of real numbers is uncountable 

computable function: a function for which there is a com- 
puter program in some programming language that finds its 
values 

uncomputable function: a function for which no computer 
program in a programming language exists that finds its 
values 

continuum hypothesis: the statement there no set A exists 
such that Xo < |A| <c 

matrix: a rectangular array of numbers 

matrix addition: see page 178 

matrix multiplication: see page 179 

I,, (identity matrix of order n): the n x n matrix that has 
entries equal to 1 on its diagonal and Os elsewhere 

A! (transpose of A): the matrix obtained from A by interchang- 
ing the rows and columns 

symmetric matrix: a matrix is symmetric if it equals its trans- 
pose 

zero—one matrix: a matrix with each entry equal to either 0 or 
1 

A V B (the join of A and B): see page 181 

A AB (the meet of A and B): see page 181 

A © B (the Boolean product of A and B): see page 182 


RESULTS 


The set identities given in Table 1 in Section 2.2 
The summation formulae in Table 2 in Section 2.4 
The set of rational numbers is countable. 

The set of real numbers is uncountable. 


5. a) Define the union, intersection, difference, and sym- 
metric difference of two sets. 


b) What are the union, intersection, difference, and sym- 
metric difference of the set of positive integers and the 
set of odd integers? 


6. a) Explain what it means for two sets to be equal. 


b) Describe as many of the ways as you can to show that 
two sets are equal. 


c) Show in at least two different ways that the sets 
A —(BNC) and (A — B) U(A — C) are equal. 


7. 


8. 


Explain the relationship between logical equivalences and 

set identities. 

a) Define the domain, codomain, and range of a function. 

b) Let f (1) be the function from the set of integers to the 
set of integers such that f(n) = n” + 1. What are the 
domain, codomain, and range of this function? 


. a) Define what it means for a function from the set of 


positive integers to the set of positive integers to be 
one-to-one. 

b) Define what it means for a function from the set of 
positive integers to the set of positive integers to be 
onto. 

c) Give an example of a function from the set of posi- 
tive integers to the set of positive integers that is both 
one-to-one and onto. 

d) Give an example of a function from the set of positive 
integers to the set of positive integers that is one-to-one 
but not onto. 

e) Give an example of a function from the set of posi- 
tive integers to the set of positive integers that is not 
one-to-one but is onto. 

f) Give an example of a function from the set of positive 
integers to the set of positive integers that is neither 
one-to-one nor onto. 


Supplementary Exercises 


1. 


Let A be the set of English words that contain the letter 

x, and let B be the set of English words that contain the 

letter g. Express each of these sets as a combination of A 

and B. 

a) The set of English words that do not contain the letter 
Xx. 

b) The set of English words that contain both an x anda 
q. 

c) The set of English words that contain an x but notaq. 

d) The set of English words that do not contain either an 
x oragd. 

e) The set of English words that contain an x or a q, but 
not both. 


. Show that if A is a subset of B, then the power set of A 


is a subset of the power set of B. 


. Suppose that A and B are sets such that the power set of 


A is a subset of the power set of B. Does it follow that A 
is a subset of B? 


. Let E denote the set of even integers and O denote the 


set of odd integers. As usual, let Z denote the set of all 
integers. Determine each of these sets. 


a)EUO b) ENO ce) Z—E d)Z—O 


. Show that if A and B are sets, then A — (A — B) = 


AN B. 


. Let A and B be sets. Show that A C B if and only if 


ANB=A. 


10. 


11. 


12. 


13. 


14. 


15. 
16. 
17. 


18. 


12. 


13. 


14. 
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a) Define the inverse of a function. 
b) When does a function have an inverse? 


c) Does the function f(n) = 10 — n from the set of inte- 
gers to the set of integers have an inverse? If so, what 
is it? 

a) Define the floor and ceiling functions from the set of 
real numbers to the set of integers. 


b) For which real numbers x is it true that |x] = [x]? 


Conjecture a formula for the terms of the sequence that 
begins 8, 14, 32, 86, 248 and find the next three terms of 
your sequence. 


Suppose that a, = ad,_, — 5 forn = 1, 2,.... Finda for- 
mula for ay. 


What is the sum of the terms of the geometric progression 
a+ar+---+ar" whenr 4 1? 


Show that the set of odd integers is countable. 
Give an example of an uncountable set. 


Define the product of two matrices A and B. When is this 
product defined? 


Show that matrix multiplication is not commutative. 


. Let A, B, and C be sets. Show that (A — B) — C is not 


necessarily equal to A — (B — C). 


. Suppose that A, B, and C are sets. Prove or disprove that 


(A—B)—-C=(A-C)-B. 


. Suppose that A, B, C, and D are sets. Prove or disprove 


that (A — B) 


(C —D)=(A-C)-(B-D). 


. Show that if A and B are finite sets, then |AM B| < 


|A U B|. Determine when this relationship is an equality. 


. Let A and B be sets in a finite universal set U. List the 


following in order of increasing size. 

a) |Al,|AU BI, |ANM BI, |UI, |D| 

b) |A— BI, |A @ BI, |A| + |B], |AU BI, |B} 

Let A and B be subsets of the finite universal set U. Show 

that |AM B| = |U| — |A| — |B] + |AN Bl. 

Let f and g be functions from {1, 2, 3, 4} to {a, b, c, d} 

and from {a,b,c,d} to {1,2,3,4}, respectively, with 

fM=d, fQ=c, fB)=a, and f(4)=)5, and 

g(a) = 2, g(b) = 1, g(c) = 3, and g(d) = 2. 

a) Is f one-to-one? Is g one-to-one? 

b) Is f onto? Is g onto? 

c) Does either f or g have an inverse? If so, find this 
inverse. 


Suppose that f is a function from A to B where A and B 
are finite sets. Explain why | f(S)| < |S| for all subsets S$ 
of A. 
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15. 


Suppose that f is a function from A to B where A and B 
are finite sets. Explain why | f(S)| = || for all subsets § 
of A if and only if f is one-to-one. 


Suppose that f is a function from A to B. We define the func- 
tion Sy from P(A) to P(B) by the rule S¢(X) = f(X) for 
each subset X of A. Similarly, we define the function S fo 
from P(B) to P(A) by the rule S/-1(Y) = f—!(Y) for each 
subset Y of B. Here, we are using Definition 4, and the defi- 
nition of the inverse image of a set found in the preamble to 
Exercise 42, both in Section 2.3. 


*16. 


17. 


18. 


19. 


20. 


21. 


22. 
23. 


24. 
25. 


26. 


* 27, 


Suppose that f is a function from the set A to the set B. 

Prove that 

a) if f is one-to-one, then Sf is a one-to-one function 
from P(A) to P(B). 

b) if f is onto function, then S+ is an onto function from 
P(A) to P(B). 

c) if f is onto function, then $ fo is a one-to-one func- 
tion from P(B) to P(A). 

d) if f is one-to-one, then S fol is an onto function from 
P(B) to P(A). 

e) if f is a one-to-one correspondence, then Sf is a one- 
to-one correspondence from P(A) to P(B) and S fol 
is a One-to-one correspondence from P(B) to P(A). 
[Hint: Use parts (a)-(d).] 

Prove that if f and g are functions from A to B and 

Sy = Sg (using the definition in the preamble to Exercise 

16), then f(x) = g(x) forall x € A. 

Show that if 1 is an integer, thenn = [n/2] + [n/2]. 

For which real numbers x and y is it true that |x + y] = 

Lx] + Ly]? 

For which real numbers x and y is it true that [x + y] = 

[x] + fy? 

For which real numbers x and y is it true that [x + y] = 

[x1 + Ly]? 

Prove that |n/2|[n/2] = |n2/4] for all integers n. 

Prove that if m is an integer, then |x|] + |m—x] = 

m — 1, unless x is an integer, in which case, it equals m. 

Prove that if x is areal number, then [|x /2|/2] = |x/4]. 

Prove that if n is an odd integer, then [n?/4] = 

(n? + 3)/4. 

Prove that if m and n are positive integers and x is a real 

number, then 


IxJ+n| |x+n 
m ~ m | 
Prove that if m is a positive integer and x is areal number, 
then 


* 28. 


29. 


* 30. 


*31. 


32. 


33. 


34. 


*% 35, 


** 36. 


37. 


38. 


39. 


40. 


41. 


We define the Ulam numbers by setting vu; = 1 and 
uz = 2. Furthermore, after determining whether the in- 
tegers less than n are Ulam numbers, we set n equal to 
the next Ulam number if it can be written uniquely as the 
sum of two different Ulam numbers. Note that u3 = 3, 
ug = 4, us = 6, and ug = 8. 

a) Find the first 20 Ulam numbers. 

b) Prove that there are infinitely many Ulam numbers. 


Determine the value of te ae (The notation used 


here for products is defined in the preamble to Exercise 
43 in Section 2.4.) 


Determine a rule for generating the terms of the sequence 
that begins 1, 3, 4, 8, 15, 27, 50, 92, .. ., and find the next 
four terms of the sequence. 


Determine a rule for generating the terms of the sequence 
that begins 2,3,3,5,10, 13,39, 43, 172, 177, 885, 
891, ..., and find the next four terms of the sequence. 
Show that the set of irrational numbers is an uncountable 
set. 


Show that the set S is a countable set if there is a func- 
tion f from S to the positive integers such that f—!(/) is 
countable whenever j is a positive integer. 


Show that the set of all finite subsets of the set of positive 
integers is a countable set. 


Show that |R x R| =|R|. [Hint: Use the Schréder- 
Bernstein theorem to show that |(0,1) x (0, 1)| = 
|(O, 1)|. To construct an injection from (0, 1) x (0, 1) to 
(0, 1), suppose that (x, y) € (0, 1) x (0, 1). Map (x, y) 
to the number with decimal expansion formed by alter- 
nating between the digits in the decimal expansions of x 
and y, which do not end with an infinite string of 9s.] 
Show that C, the set of complex numbers has the same 
cardinality as R, the set of real numbers. 

Find A” if A is 


Show that if A = cI, where c is areal number and I is the 
n X n identity matrix, then AB = BA whenever B is an 
n X n matrix. 

Show thatif Aisa2 x 2 matrix such that AB = BA when- 
ever B is a 2 x 2 matrix, then A = cI, where c is a real 
number and I is the 2 x 2 identity matrix. 

Show that if A and B are invertible matrices and AB exists, 
then (AB)~! = B-!A7!. 

Let A be ann x n matrix and let 0 be the n x n matrix 
all of whose entries are zero. Show that the following are 
true. 

a) AO0=00A=0 

b) AVO=0VA=A 

c) AAOD=0AA=0 


Computer Projects 


Write programs with the specified input and output. 


. Given subsets A and B of a set with n elements, use bit 
strings to find A, AUB, ANB, A— B, and A® B. 


. Given multisets A and B from the same universal set, find 
AU B,ANB,A-— B, and A + B (see preamble to Exer- 
cise 61 of Section 2.2). 


. Given fuzzy sets A and B, find A, AUB, and AN B (see 
preamble to Exercise 63 of Section 2.2). 

. Givena function f from {1, 2, ...,} to the set of integers, 
determine whether f is one-to-one. 


. Given a function f from {1,2,...,} to itself, determine 
whether f is onto. 


Computations and Explorations 


. Given two finite sets, list all elements in the Cartesian prod- 
uct of these two sets. 

. Given a finite set, list all elements of its power set. 

. Calculate the number of one-to-one functions from a set $ 
toaset 7, where S and T are finite sets of various sizes. Can 
you determine a formula for the number of such functions? 
(We will find such a formula in Chapter 6.) 

. Calculate the number of onto functions from a set S to a 
set T, where S and T are finite sets of various sizes. Can 


Writing Projects 


Respond to these with essays using outside sources. 


. Discuss how an axiomatic set theory can be developed to 
avoid Russell’s paradox. (See Exercise 46 of Section 2.1.) 


. Research where the concept of a function first arose, and 
describe how this concept was first used. 


. Explain the different ways in which the Encyclopedia of 
Integer Sequences has been found useful. Also, describe 
a few of the more unusual sequences in this encyclopedia 
and how they arise. 


11. 


12. 


#5, 


4. 


Computer Projects 189 


. Given a bijection f from the set {1, 2, ..., m} to itself, find 
a 
. Given anm x k matrix A and ak x n matrix B, find AB. 


. Given a square matrix A and a positive integer n, find A”. 
. Given a square matrix, determine whether it is symmetric. 
. Given two m x n Boolean matrices, find their meet and 


join. 
Given an m x k Boolean matrix A and a k x n Boolean 
matrix B, find the Boolean product of A and B. 


Given a square Boolean matrix A and a positive integer n, 
find Al”), 


Use a computational program or programs you have written to do these exercises. 


you determine a formula for the number of such functions? 
(We will find such a formula in Chapter 8.) 


Develop a collection of different rules for generating the 
terms of a sequence and a program for randomly selecting 
one of these rules and the particular sequence generated 
using these rules. Make this part of an interactive program 
that prompts for the next term of the sequence and deter- 
mines whether the response is the intended next term. 


Define the recently invented EKG sequence and describe 
some of its properties and open questions about it. 


. Look up the definition of a transcendental number. Explain 


how to show that such numbers exist and how such num- 
bers can be constructed. Which famous numbers can be 
shown to be transcendental and for which famous numbers 
is it still unknown whether they are transcendental? 


. Expand the discussion of the continuum hypothesis in the 


text. 


3.1 Algorithms 


3.2 The Growth of 
Functions 


3.3 Complexity of 
Algorithms 


Algorithms 


M any problems can be solved by considering them as special cases of general problems. 
For instance, consider the problem of locating the largest integer in the sequence 101, 
12, 144, 212, 98. This is a specific case of the problem of locating the largest integer in a sequence 
of integers. To solve this general problem we must give an algorithm, which specifies a sequence 
of steps used to solve this general problem. We will study algorithms for solving many different 
types of problems in this book. For example, in this chapter we will introduce algorithms for 
two of the most important problems in computer science, searching for an element in a list and 
sorting a list so its elements are in some prescribed order, such as increasing, decreasing, or 
alphabetic. Later in the book we will develop algorithms that find the greatest common divisor 
of two integers, that generate all the orderings of a finite set, that find the shortest path between 
nodes in a network, and for solving many other problems. 

We will also introduce the notion of an algorithmic paradigm, which provides a general 
method for designing algorithms. In particular we will discuss brute-force algorithms, which 
find solutions using a straightforward approach without introducing any cleverness. We will also 
discuss greedy algorithms, a class of algorithms used to solve optimization problems. Proofs are 
important in the study of algorithms. In this chapter we illustrate this by proving that a particular 
greedy algorithm always finds an optimal solution. 

One important consideration concerning an algorithm is its computational complexity, 
which measures the processing time and computer memory required by the algorithm to solve 
problems of a particular size. To measure the complexity of algorithms we use big-O and big- 
Theta notation, which we develop in this chapter. We will illustrate the analysis of the complexity 
of algorithms in this chapter, focusing on the time an algorithm takes to solve a problem. Fur- 
thermore, we will discuss what the time complexity of an algorithm means in practical and 
theoretical terms. 


Algorithms 


DEFINITION 1 


Introduction 


There are many general classes of problems that arise in discrete mathematics. For instance: 
given a sequence of integers, find the largest one; given a set, list all its subsets; given a set 
of integers, put them in increasing order; given a network, find the shortest path between two 
vertices. When presented with such a problem, the first thing to do is to construct a model that 
translates the problem into a mathematical context. Discrete structures used in such models 
include sets, sequences, and functions—structures discussed in Chapter 2—as well as such 
other structures as permutations, relations, graphs, trees, networks, and finite state machines— 
concepts that will be discussed in later chapters. 

Setting up the appropriate mathematical model is only part of the solution. To complete the 
solution, a method is needed that will solve the general problem using the model. Ideally, what 
is required is a procedure that follows a sequence of steps that leads to the desired answer. Such 
a sequence of steps is called an algorithm. 


An algorithm is a finite sequence of precise instructions for performing a computation or for 
solving a problem. 
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The term algorithm is a corruption of the name al-Khowarizmi, a mathematician of the ninth 
century, whose book on Hindu numerals is the basis of modern decimal notation. Originally, 
the word algorism was used for the rules for performing arithmetic using decimal notation. 
Algorism evolved into the word algorithm by the eighteenth century. With the growing interest 
in computing machines, the concept of an algorithm was given a more general meaning, to 
include all definite procedures for solving problems, not just the procedures for performing 
arithmetic. (We will discuss algorithms for performing arithmetic with integers in Chapter 4.) 

In this book, we will discuss algorithms that solve a wide variety of problems. In this 
section we will use the problem of finding the largest integer in a finite sequence of integers to 
illustrate the concept of an algorithm and the properties algorithms have. Also, we will describe 
algorithms for locating a particular element in a finite set. In subsequent sections, procedures 
for finding the greatest common divisor of two integers, for finding the shortest path between 
two points in a network, for multiplying matrices, and so on, will be discussed. 


Describe an algorithm for finding the maximum (largest) value in a finite sequence of integers. 


Even though the problem of finding the maximum element in a sequence is relatively trivial, 
it provides a good illustration of the concept of an algorithm. Also, there are many instances 
where the largest integer in a finite sequence of integers is required. For instance, a university 
may need to find the highest score on a competitive exam taken by thousands of students. Or 
a sports organization may want to identify the member with the highest rating each month. 
We want to develop an algorithm that can be used whenever the problem of finding the largest 
element in a finite sequence of integers arises. 

We can specify a procedure for solving this problem in several ways. One method is simply to 
use the English language to describe the sequence of steps used. We now provide such a solution. 


Solution of Example 1: We perform the following steps. 


1. Set the temporary maximum equal to the first integer in the sequence. (The temporary 
maximum will be the largest integer examined at any stage of the procedure.) 


2. Compare the next integer in the sequence to the temporary maximum, and if it is larger 
than the temporary maximum, set the temporary maximum equal to this integer. 


3. Repeat the previous step if there are more integers in the sequence. 


4. Stop when there are no integers left in the sequence. The temporary maximum at this 
point is the largest integer in the sequence. < 


An algorithm can also be described using a computer language. However, when that is done, 
only those instructions permitted in the language can be used. This often leads to a description 
of the algorithm that is complicated and difficult to understand. Furthermore, because many 
programming languages are in common use, it would be undesirable to choose one particular 
language. So, instead of using a particular computer language to specify algorithms, a form 
of pseudocode, described in Appendix 3, will be used in this book. (We will also describe 
algorithms using the English language.) Pseudocode provides an intermediate step between 


ABU JA‘FAR MOHAMMED IBN MUSA AL-KHOWARIZMI (C. 780—C. 850) al-Khowarizmi, an as- 
tronomer and mathematician, was a member of the House of Wisdom, an academy of scientists in Baghdad. 
The name al-Khowarizmi means “from the town of Kowarzizm,” which was then part of Persia, but is now 
called Khiva and is part of Uzbekistan. al-Khowarizmi wrote books on mathematics, astronomy, and geography. 
Western Europeans first learned about algebra from his works. The word algebra comes from al-jabr, part of 
the title of his book Kitab al-jabr w’al muquabala. This book was translated into Latin and was a widely used 
textbook. His book on the use of Hindu numerals describes procedures for arithmetic operations using these 
numerals. European authors used a Latin corruption of his name, which later evolved to the word algorithm, to 
describe the subject of arithmetic with Hindu numerals. 


EXAMPLE 2 


3.1 Algorithms 193 


an English language description of an algorithm and an implementation of this algorithm in a 
programming language. The steps of the algorithm are specified using instructions resembling 
those used in programming languages. However, in pseudocode, the instructions used can 
include any well-defined operations or statements. A computer program can be produced in 
any computer language using the pseudocode description as a starting point. 

The pseudocode used in this book is designed to be easily understood. It can serve as an 
intermediate step in the construction of programs implementing algorithms in one of a variety of 
different programming languages. Although this pseudocode does not follow the syntax of Java, 
C, C++, or any other programming language, students familiar with a modern programming 
language will find it easy to follow. A key difference between this pseudocode and code in a 
programming language is that we can use any well-defined instruction even if it would take 
many lines of code to implement this instruction. The details of the pseudocode used in the text 
are given in Appendix 3. The reader should refer to this appendix whenever the need arises. 

A pseudocode description of the algorithm for finding the maximum element in a finite 
sequence follows. 


ALGORITHM 1 Finding the Maximum Element in a Finite Sequence. 


procedure max(a),d2,..., Qn: integers) 
max := a, 
for i :=2ton 
if max < a; then max := a; 
return max{max is the largest element} 


This algorithm first assigns the initial term of the sequence, a1, to the variable max. The “for” 
loop is used to successively examine terms of the sequence. If a term is greater than the current 
value of max, it is assigned to be the new value of max. 


PROPERTIES OF ALGORITHMS There are several properties that algorithms generally 
share. They are useful to keep in mind when algorithms are described. These properties are: 


@ Input. An algorithm has input values from a specified set. 

® Output. From each set of input values an algorithm produces output values from a spec- 
ified set. The output values are the solution to the problem. 

& Definiteness. The steps of an algorithm must be defined precisely. 

® Correctness. An algorithm should produce the correct output values for each set of input 
values. 

& Finiteness. An algorithm should produce the desired output after a finite (but perhaps 
large) number of steps for any input in the set. 

= Effectiveness. It must be possible to perform each step of an algorithm exactly and in a 
finite amount of time. 

= Generality. The procedure should be applicable for all problems of the desired form, not 
just for a particular set of input values. 


Show that Algorithm 1 for finding the maximum element in a finite sequence of integers has all 
the properties listed. 


Solution: The input to Algorithm | is a sequence of integers. The output is the largest integer 
in the sequence. Each step of the algorithm is precisely defined, because only assignments, a 
finite loop, and conditional statements occur. To show that the algorithm is correct, we must 
show that when the algorithm terminates, the value of the variable max equals the maximum 
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of the terms of the sequence. To see this, note that the initial value of max is the first term of 
the sequence; as successive terms of the sequence are examined, max is updated to the value 
of a term if the term exceeds the maximum of the terms previously examined. This (informal) 
argument shows that when all the terms have been examined, max equals the value of the largest 
term. (A rigorous proof of this requires techniques developed in Section 5.1.) The algorithm 
uses a finite number of steps, because it terminates after all the integers in the sequence have 
been examined. The algorithm can be carried out in a finite amount of time because each step is 
either a comparison or an assignment, there are a finite number of these steps, and each of these 
two operations takes a finite amount of time. Finally, Algorithm 1 is general, because it can be 
used to find the maximum of any finite sequence of integers. < 


Searching Algorithms 


The problem of locating an element in an ordered list occurs in many contexts. For instance, a 
program that checks the spelling of words searches for them in a dictionary, which is just an 
ordered list of words. Problems of this kind are called searching problems. We will discuss 
several algorithms for searching in this section. We will study the number of steps used by each 
of these algorithms in Section 3.3. 

The general searching problem can be described as follows: Locate an element x in a list of 
distinct elements a1, a2, ..., @,, or determine that it is not in the list. The solution to this search 
problem is the location of the term in the list that equals x (that is, 7 is the solution if x = aj) 
and is 0 if x is not in the list. 


THE LINEAR SEARCH The first algorithm that we will present is called the linear search, 
or sequential search, algorithm. The linear search algorithm begins by comparing x and ay. 
When x = ay, the solution is the location of a;, namely, 1. When x 4 a}, compare x with az. If 
xX = az, the solution is the location of a2, namely, 2. When x 4 a2, compare x with a3. Continue 
this process, comparing x successively with each term of the list until a match is found, where 
the solution is the location of that term, unless no match occurs. If the entire list has been 
searched without locating x, the solution is 0. The pseudocode for the linear search algorithm 
is displayed as Algorithm 2. 


ALGORITHM 2 The Linear Search Algorithm. 


procedure /Jinear search(x: integer, a), a2,...,@,: distinct integers) 
iZ=1 
while (i <n andx 4 a;) 
i:=i+1 
if i < n then location := i 
else location := 0 
return Jocation{ location is the subscript of the term that equals x, or is 0 if x is not found} 


THE BINARY SEARCH We will now consider another searching algorithm. This algorithm 
can be used when the list has terms occurring in order of increasing size (for instance: if the 
terms are numbers, they are listed from smallest to largest; if they are words, they are listed 
in lexicographic, or alphabetic, order). This second searching algorithm is called the binary 
search algorithm. It proceeds by comparing the element to be located to the middle term of 
the list. The list is then split into two smaller sublists of the same size, or where one of these 
smaller lists has one fewer term than the other. The search continues by restricting the search 
to the appropriate sublist based on the comparison of the element to be located and the middle 
term. In Section 3.3, it will be shown that the binary search algorithm is much more efficient 
than the linear search algorithm. Example 3 demonstrates how a binary search works. 


EXAMPLE 3 
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To search for 19 in the list 
1235678 10 12 13 15 16 18 19 20 22, 

first split this list, which has 16 terms, into two smaller lists with eight terms each, namely, 
123567810 12 13 15 16 18 19 20 22. 


Then, compare 19 and the largest term in the first list. Because 10 < 19, the search for 19 can 
be restricted to the list containing the 9th through the 16th terms of the original list. Next, split 
this list, which has eight terms, into the two smaller lists of four terms each, namely, 


12 13 15 16 18 19 20 22. 


Because 16 < 19 (comparing 19 with the largest term of the first list) the search is restricted to 
the second of these lists, which contains the 13th through the 1 6th terms of the original list. The 
list 18 19 20 22 is split into two lists, namely, 


18 19 20 22. 


Because 19 is not greater than the largest term of the first of these two lists, which is also 19, the 
search is restricted to the first list: 18 19, which contains the 13th and 14th terms of the original 
list. Next, this list of two terms is split into two lists of one term each: 18 and 19. Because 
18 < 19, the search is restricted to the second list: the list containing the 14th term of the list, 
which is 19. Now that the search has been narrowed down to one term, a comparison is made, 
and 19 is located as the 14th term in the original list. 


We now specify the steps of the binary search algorithm. To search for the integer x in the 
list a], a2, ..., An, where aj < az < +--+ < dy, begin by comparing x with the middle term a;, 
of the list, where m = [(n + 1)/2]. (Recall that |x| is the greatest integer not exceeding x.) If 
xX > dm, the search for x is restricted to the second half of the list, which is ay41, Gm42,---;n- 
If x is not greater than a,,, the search for x is restricted to the first half of the list, which is 
a1, 42,...,4m. 

The search has now been restricted to a list with no more than [1/2] elements. (Recall that 
[x] is the smallest integer greater than or equal to x.) Using the same procedure, compare x to 
the middle term of the restricted list. Then restrict the search to the first or second half of the 
list. Repeat this process until a list with one term is obtained. Then determine whether this term 
is x. Pseudocode for the binary search algorithm is displayed as Algorithm 3. 


ALGORITHM 3 The Binary Search Algorithm. 


procedure binary search (x: integer, a1, a2,..., dn: increasing integers) 
i := L{i is left endpoint of search interval} 
j :=n {j is right endpoint of search interval} 
while i < j 
m:= (i+ j)/21 
if x > a, theni:=m+1 
else j :=m 
if x = a; then location := i 
else location := 0 
return location{location is the subscript i of the term a; equal to x, or 0 if x is not found} 
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Algorithm 3 proceeds by successively narrowing down the part of the sequence being 
searched. At any given stage only the terms from a; to aj; are under consideration. In other 
words, i and j are the smallest and largest subscripts of the remaining terms, respectively. 
Algorithm 3 continues narrowing the part of the sequence being searched until only one term 
of the sequence remains. When this is done, a comparison is made to see whether this term 
equals x. 


Sorting 


Ordering the elements of a list is a problem that occurs in many contexts. For example, to produce 
atelephone directory it is necessary to alphabetize the names of subscribers. Similarly, producing 
a directory of songs available for downloading requires that their titles be put in alphabetic order. 
Putting addresses in order in an e-mail mailing list can determine whether there are duplicated 
addresses. Creating a useful dictionary requires that words be put in alphabetical order. Similarly, 
generating a parts list requires that we order them according to increasing part number. 

Suppose that we have a list of elements of a set. Furthermore, suppose that we have a way to 
order elements of the set. (The notion of ordering elements of sets will be discussed in detail in 
Section 9.6.) Sorting is putting these elements into a list in which the elements are in increasing 
order. For instance, sorting the list 7, 2, 1, 4, 5, 9 produces the list 1, 2, 4, 5, 7, 9. Sorting the 
list d, h, c, a, f (using alphabetical order) produces the list a, c, d, f, h. 

An amazingly large percentage of computing resources is devoted to sorting one thing 
or another. Hence, much effort has been devoted to the development of sorting algorithms. 
A surprisingly large number of sorting algorithms have been devised using distinct strate- 
gies, with new ones introduced regularly. In his fundamental work, The Art of Computer 
Programming, Donald Knuth devotes close to 400 pages to sorting, covering around 15 
different sorting algorithms in depth! More than 100 sorting algorithms have been de- 
vised, and it is surprising how often new sorting algorithms are developed. Among the 
newest sorting algorithms that have caught on is the the library sort, also known as the 
gapped insertion sort, invented as recently as 2006. There are many reasons why sort- 
ing algorithms interest computer scientists and mathematicians. Among these reasons are 
that some algorithms are easier to implement, some algorithms are more efficient (either 
in general, or when given input with certain characteristics, such as lists slightly out of 
order), some algorithms take advantage of particular computer architectures, and some al- 
gorithms are particularly clever. In this section we will introduce two sorting algorithms, 
the bubble sort and the insertion sort. Two other sorting algorithms, the selection sort 
and the binary insertion sort, are introduced in the exercises, and the shaker sort is in- 
troduced in the Supplementary Exercises. In Section 5.4 we will discuss the merge sort 
and introduce the quick sort in the exercises in that section; the tournament sort is in- 
troduced in the exercise set in Section 11.2. We cover sorting algorithms both because 
sorting is an important problem and because these algorithms can serve as examples 
for many important concepts. 


THE BUBBLE SORT The bubble sort is one of the simplest sorting algorithms, but not one 
of the most efficient. It puts a list into increasing order by successively comparing adjacent 
elements, interchanging them if they are in the wrong order. To carry out the bubble sort, we 
perform the basic operation, that is, interchanging a larger element with a smaller one following 
it, starting at the beginning of the list, for a full pass. We iterate this procedure until the sort is 
complete. Pseudocode for the bubble sort is given as Algorithm 4. We can imagine the elements 
in the list placed in a column. In the bubble sort, the smaller elements “bubble” to the top as 
they are interchanged with larger elements. The larger elements “sink” to the bottom. This is 
illustrated in Example 4. 


Use the bubble sort to put 3, 2, 4, 1, 5 into increasing order. 
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guaranteed to be in correct order 


FIGURE 1 The Steps of a Bubble Sort. 


Solution: The steps of this algorithm are illustrated in Figure 1. Begin by comparing the first two 
elements, 3 and 2. Because 3 > 2, interchange 3 and 2, producing the list 2, 3, 4, 1, 5. Because 
3 < 4, continue by comparing 4 and |. Because 4 > 1, interchange | and 4, producing the list 
2,3, 1,4, 5. Because 4 < 5, the first pass is complete. The first pass guarantees that the largest 
element, 5, is in the correct position. 

The second pass begins by comparing 2 and 3. Because these are in the correct order, 3 and | 
are compared. Because 3 > 1, these numbers are interchanged, producing 2, 1, 3, 4, 5. Because 
3 < 4, these numbers are in the correct order. It is not necessary to do any more comparisons 
for this pass because 5 is already in the correct position. The second pass guarantees that the 
two largest elements, 4 and 5, are in their correct positions. 

The third pass begins by comparing 2 and |. These are interchanged because 2 > 1, produc- 
ing 1, 2,3, 4,5. Because 2 < 3, these two elements are in the correct order. It is not necessary to 
do any more comparisons for this pass because 4 and 5 are already in the correct positions. The 
third pass guarantees that the three largest elements, 3, 4, and 5, are in their correct positions. 

The fourth pass consists of one comparison, namely, the comparison of | and 2. Because 
1 < 2, these elements are in the correct order. This completes the bubble sort. < 


ALGORITHM 4 The Bubble Sort. 


procedure bubblesort(a,...,@, : real numbers with n > 2) 
fori :=1ton—1 
for j := 1ton—i 
if a; > aj+, then interchange a; and aj+1 
{a,..., Gp 18 in increasing order} 


THE INSERTION SORT The insertion sort is a simple sorting algorithm, but it is usually 
not the most efficient. To sort a list with n elements, the insertion sort begins with the second 
element. The insertion sort compares this second element with the first element and inserts it 
before the first element if it does not exceed the first element and after the first element if it 
exceeds the first element. At this point, the first two elements are in the correct order. The third 
element is then compared with the first element, and if it is larger than the first element, it is 
compared with the second element; it is inserted into the correct position among the first three 
elements. 

In general, in the jth step of the insertion sort, the jth element of the list is inserted into 
the correct position in the list of the previously sorted 7 — 1 elements. To insert the jth element 
in the list, a linear search technique is used (see Exercise 43); the jth element is successively 
compared with the already sorted j — 1 elements at the start of the list until the first element that 
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is not less than this element is found or until it has been compared with all j — 1 elements; the jth 
element is inserted in the correct position so that the first 7 elements are sorted. The algorithm 
continues until the last element is placed in the correct position relative to the already sorted list 
of the first n — 1 elements. The insertion sort is described in pseudocode in Algorithm 5. 


Use the insertion sort to put the elements of the list 3, 2, 4, 1, 5 in increasing order. 


Solution: The insertion sort first compares 2 and 3. Because 3 > 2, it places 2 in the first position, 
producing the list 2, 3, 4, 1, 5 (the sorted part of the list is shown in color). At this point, 2 and 3 
are in the correct order. Next, it inserts the third element, 4, into the already sorted part of the list 
by making the comparisons 4 > 2 and 4 > 3. Because 4 > 3, 4 remains in the third position. 
At this point, the list is 2, 3, 4, 1, 5 and we know that the ordering of the first three elements 
is correct. Next, we find the correct place for the fourth element, 1, among the already sorted 
elements, 2,3, 4. Because | < 2, we obtain the list |, 2, 3, 4, 5. Finally, we insert 5 into the 
correct position by successively comparing it to 1, 2, 3, and 4. Because 5 > 4, it stays at the end 
of the list, producing the correct order for the entire list. < 


ALGORITHM 5 The Insertion Sort. 


procedure insertion sort(a\, a2,..., dn: real numbers with n > 2) 
for j :=2ton 
a | 
while aj > aj 
i:=i+l1 
m:= aj 
fork :=Otoj —i-1 
Aj—k = Aj—k-1 


qj :=m 
{a,, ..., Gy 1S in increasing order} 
Greedy Algorithms 


Many algorithms we will study in this book are designed to solve optimization problems. 
The goal of such problems is to find a solution to the given problem that either minimizes or 
maximizes the value of some parameter. Optimization problems studied later in this text include 
finding a route between two cities with smallest total mileage, determining a way to encode 
messages using the fewest bits possible, and finding a set of fiber links between network nodes 
using the least amount of fiber. 

Surprisingly, one of the simplest approaches often leads to a solution of an optimization 
problem. This approach selects the best choice at each step, instead of considering all sequences 
of steps that may lead to an optimal solution. Algorithms that make what seems to be the “best” 
choice at each step are called greedy algorithms. Once we know that a greedy algorithm finds a 
feasible solution, we need to determine whether it has found an optimal solution. (Note that we 
call the algoritm “greedy” whether or not it finds an optimal solution.) To do this, we either prove 
that the solution is optimal or we show that there is a counterexample where the algorithm yields 
a nonoptimal solution. To make these concepts more concrete, we will consider an algorithm 
that makes change using coins. 
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Consider the problem of making n cents change with quarters, dimes, nickels, and pennies, and 
using the least total number of coins. We can devise a greedy algorithm for making change for 
n cents by making a locally optimal choice at each step; that is, at each step we choose the coin 
of the largest denomination possible to add to the pile of change without exceeding n cents. For 
example, to make change for 67 cents, we first select a quarter (leaving 42 cents). We next select 
a second quarter (leaving 17 cents), followed by a dime (leaving 7 cents), followed by a nickel 
(leaving 2 cents), followed by a penny (leaving | cent), followed by a penny. < 


We display a greedy change-making algorithm for n cents, using any set of denominations 
of coins, as Algorithm 6. 


ALGORITHM 6 Greedy Change-Making Algorithm. 


procedure change(ci, c2,..., Cr: values of denominations of coins, where 
cj > C2 >-+++ > Cy; ni a positive integer) 
fori := 1 tor 
d; := 0 {dj counts the coins of denomination c; used} 
while n > c; 
d; :=d; + 1 {add a coin of denomination c; } 
Nni=n—C 
{d; is the number of coins of denomination c; in the change fori = 1,2,...,r} 


We have described a greedy algorithm for making change using any finite set of coins with 
denominations c1, C2, ..., c. In the particular case where the four denominations are quarters 
dimes, nickels, and pennies, we have c; = 25, cz = 10, c3 = 5, and cq = 1. For this case, we 
will show that this algorithm leads to an optimal solution in the sense that it uses the fewest 
coins possible. Before we embark on our proof, we show that there are sets of coins for which 
the greedy algorithm (Algorithm 6) does not necessarily produce change using the fewest coins 
possible. For example, if we have only quarters, dimes, and pennies (and no nickels) to use, 
the greedy algorithm would make change for 30 cents using six coins—a quarter and five 
pennies—whereas we could have used three coins, namely, three dimes. 


If n is a positive integer, then n cents in change using quarters, dimes, nickels, and pennies 
using the fewest coins possible has at most two dimes, at most one nickel, at most four 
pennies, and cannot have two dimes and a nickel. The amount of change in dimes, nickels, 
and pennies cannot exceed 24 cents. 


Proof: We use a proof by contradiction. We will show that if we had more than the specified 
number of coins of each type, we could replace them using fewer coins that have the same value. 
We note that if we had three dimes we could replace them with a quarter and a nickel, if we 
had two nickels we could replace them with a dime, if we had five pennies we could replace 
them with a nickel, and if we had two dimes and a nickel we could replace them with a quarter. 
Because we can have at most two dimes, one nickel, and four pennies, but we cannot have two 
dimes and a nickel, it follows that 24 cents is the most money we can have in dimes, nickels, 
and pennies when we make change using the fewest number of coins for 7 cents. 4 


The greedy algorithm (Algorithm 6) produces change using the fewest coins possible. 
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Proof: We will use a proof by contradiction. Suppose that there is a positive integer n such that 
there is a way to make change for n cents using quarters, dimes, nickels, and pennies that uses 
fewer coins than the greedy algorithm finds. We first note that g’, the number of quarters used 
in this optimal way to make change for n cents, must be the same as g, the number of quarters 
used by the greedy algorithm. To show this, first note that the greedy algorithm uses the most 
quarters possible, so q’ < g. However, it is also the case that g’ cannot be less than q. If it were, 
we would need to make up at least 25 cents from dimes, nickels, and pennies in this optimal 
way to make change. But this is impossible by Lemma 1. 

Because there must be the same number of quarters in the two ways to make change, the 
value of the dimes, nickels, and pennies in these two ways must be the same, and these coins 
are worth no more than 24 cents. There must be the same number of dimes, because the greedy 
algorithm used the most dimes possible and by Lemma 1, when change is made using the fewest 
coins possible, at most one nickel and at most four pennies are used, so that the most dimes 
possible are also used in the optimal way to make change. Similarly, we have the same number 
of nickels and, finally, the same number of pennies. | 


A greedy algorithm makes the best choice at each step according to a specified criterion. 
The next example shows that it can be difficult to determine which of many possible criteria to 
choose. 


Suppose we have a group of proposed talks with preset start and end times. Devise a greedy 
algorithm to schedule as many of these talks as possible in a lecture hall, under the assumptions 
that once a talk starts, it continues until it ends, no two talks can proceed at the same time, and 
a talk can begin at the same time another one ends. Assume that talk j begins at time s; (where 
s stands for start) and ends at time e; (where e stands for end). 


Solution: To use a greedy algorithm to schedule the most talks, that is, an optimal schedule, we 
need to decide how to choose which talk to add at each step. There are many criteria we could 
use to select a talk at each step, where we chose from the talks that do not overlap talks already 
selected. For example, we could add talks in order of earliest start time, we could add talks in 
order of shortest time, we could add talks in order of earliest finish time, or we could use some 
other criterion. 

We now consider these possible criteria. Suppose we add the talk that starts earliest among 
the talks compatible with those already selected. We can construct a counterexample to see that 
the resulting algorithm does not always produce an optimal schedule. For instance, suppose that 
we have three talks: Talk | starts at 8 A.M. and ends at 12 noon, Talk 2 starts at 9 A.M. and ends 
at 10 A.M., and Talk 3 starts at 11 A.M. and ends at 12 noon. We first select the Talk 1 because it 
starts earliest. But once we have selected Talk | we cannot select either Talk 2 or Talk 3 because 
both overlap Talk 1. Hence, this greedy algorithm selects only one talk. This is not optimal 
because we could schedule Talk 2 and Talk 3, which do not overlap. 

Now suppose we add the talk that is shortest among the talks that do not overlap any of those 
already selected. Again we can construct a counterexample to show that this greedy algorithm 
does not always produce an optimal schedule. So, suppose that we have three talks: Talk | starts 
at 8 A.M. and ends at 9:15 a.m., Talk 2 starts at 9 a.m. and ends at 10 a.m., and Talk 3 starts at 
9:45 a.m. and ends at 11 A.M. We select Talk 2 because it is shortest, requiring one hour. Once 
we select Talk 2, we cannot select either Talk 1 or Talk 3 because neither is compatible with 
Talk 2. Hence, this greedy algorithm selects only one talk. However, it is possible to select two 
talks, Talk 1 and Talk 3, which are compatible. 

However, it can be shown that we schedule the most talks possible if in each step we select 
the talk with the earliest ending time among the talks compatible with those already selected. 
We will prove this in Chapter 5 using the method of mathematical induction. The first step we 
will make is to sort the talks according to increasing finish time. After this sorting, we relabel 
the talks so that e; < e2 < ... < é,. The resulting greedy algorithm is given as Algorithm 7. 4 
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ALGORITHM 7 Greedy Algorithm for Scheduling Talks. 


procedure schedule(s, < s2 < +--+ < Sp»: start times of talks, 
ey <e2 < +--+ <e,: ending times of talks) 

sort talks by finish time and reorder so that e; < e2 <... < ey 
S:=6 
for j := 1 ton 

if talk j is compatible with S then 

S:= SU {talk 7} 

return S{S is the set of talks scheduled} 


The Halting Problem 


We will now describe a proof of one of the most famous theorems in computer science. We will 
show that there is a problem that cannot be solved using any procedure. That is, we will show 
there are unsolvable problems. The problem we will study is the halting problem. It asks whether 
there is a procedure that does this: It takes as input a computer program and input to the program 
and determines whether the program will eventually stop when run with this input. It would be 
convenient to have such a procedure, if it existed. Certainly being able to test whether a program 
entered into an infinite loop would be helpful when writing and debugging programs. However, 
in 1936 Alan Turing showed that no such procedure exists (see his biography in Section 13.4). 

Before we present a proof that the halting problem is unsolvable, first note that we cannot 
simply run a program and observe what it does to determine whether it terminates when run 
with the given input. If the program halts, we have our answer, but if it is still running after any 
fixed length of time has elapsed, we do not know whether it will never halt or we just did not 
wait long enough for it to terminate. After all, it is not hard to design a program that will stop 
only after more than a billion years has elapsed. 

We will describe Turing’s proof that the halting problem is unsolvable; it is a proof by 
contradiction. (The reader should note that our proof is not completely rigorous, because we 
have not explicitly defined what a procedure is. To remedy this, the concept of a Turing machine 
is needed. This concept is introduced in Section 13.5.) 


Proof: Assume there is a solution to the halting problem, a procedure called H(P, I). The 
procedure H (P, /) takes two inputs, one a program P and the other /, an input to the program 
P. H(P,1) generates the string “halt” as output if H determines that P stops when given J as 
input. Otherwise, H (P, 7) generates the string “loops forever” as output. We will now derive a 
contradiction. 

When a procedure is coded, it is expressed as a string of characters; this string can be 
interpreted as a sequence of bits. This means that a program itself can be used as data. Therefore 
a program can be thought of as input to another program, or even itself. Hence, H can take a 
program P as both of its inputs, which are a program and input to this program. H should be 
able to determine whether P will halt when it is given a copy of itself as input. 

To show that no procedure H exists that solves the halting problem, we construct a simple 
procedure K (P), which works as follows, making use of the output H(P, P). If the output of 
H(P, P) is “loops forever,’ which means that P loops forever when given a copy of itself as 
input, then K (P) halts. If the output of H(P, P) is “halt,” which means that P halts when given 
a copy of itself as input, then K (P) loops forever. That is, K (P) does the opposite of what the 
output of H(P, P) specifies. (See Figure 2.) 

Now suppose we provide K as input to K. We note that if the output of H(K, K) is “loops 
forever,” then by the definition of K we see that K (K’) halts. Otherwise, if the output of H(K, K) 
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If H(P, P) = “halts,” 
P as program then loop forever 
ee fr | Program Gan Program = |-_____» 
npu 
P H(P, 1) P K(P) 
Program P H(P, P) 
[_- > 
P as input If H(P, P) = “loops forever,” 
then halt 
FIGURE 2 Showing that the Halting Problem is Unsolvable. 
is “halt,” then by the definition of K we see that K(K) loops forever, in violation of what H 
tells us. In both cases, we have a contradiction. 
Thus, H cannot always give the correct answers. Consequently, there is no procedure that 
solves the halting problem. | 
Exercises 
1. List all the steps used by Algorithm 1 to find the maximum 8. Describe an algorithm that takes as input a list of n dis- 
of the list 1, 8, 12, 9, 11, 2, 14, 5, 10, 4. tinct integers and finds the location of the largest even 
F : sy . integer in the list or returns 0 if there are no even integers 
2. Determine which characteristics of an algorithm de- ects ' e ; aaa 
. ; : . in the list. 

scribed in the text (after Algorithm 1) the following pro- : : : 

cedures have and which they lack. 9. A palindrome is a string that reads the same forward 

eeecie. tS and backward. Describe an algorithm for determining 

a) procedure double(n: positive integer) ? : : 

@hilans 0 whether a string of n characters is a palindrome. 
HO 10. Devise an algorithm to compute x”, where x is a real 
_ toe number and n is an integer. [Hint: First give a procedure 

b) procedure divide(n: positive integer) for computing x” when n is nonnegative by successive 

while n > 0 multiplication by x, starting with 1. Then extend this pro- 
mi=1/n cedure, and use the fact that x~” = 1/x” to compute x” 
ni=n—I when n is negative. ] 

c) procedure sum(n: positive integer) 11. Describe an algorithm that interchanges the values of the 
sum := 0 variables x and y, using only assignments. What is the 
while i < 10 minimum number of assignment statements needed to do 

sum := sum +i this? 

d) procedure choose(a, b: integers) 12. Describe an algorithm that uses only assignment state- 
x := either a or b ments that replaces the triple (x, y, z) with (y, z, x). 

3. Devise an algorithm that finds the sum of all the integers What is the minimum number of assignment statements 
in a list. needed? 

F : : : : 13. List all the steps used to search for 9 in the sequence 1, 

4. Describe an algorithm that takes as input a list of  in- ‘ ee : se 

F 3, 4, 5, 6, 8, 9, 11 using 

tegers and produces as output the largest difference ob- : : 

tained by subtracting an integer in the list from the one a) a linear search. b) a binary search. 

following it. 14. List all the steps used to search for 7 in the sequence given 

. . . ; . in Exercise 13 for both a linear search and a binary search. 
5. Describe an algorithm that takes as input a list of n inte- : F P : ‘ 
: : ; 15. Describe an algorithm that inserts an integer x in the ap- 
gers in nondecreasing order and produces the list of all : prceeae F : 
: propriate position into the list aj, a2,..., da, of integers 
values that occur more than once. (Recall that a list of ea : 

‘ : fae ' , oe that are in increasing order. 

integers is nondecreasing if each integer in the list is at ; ; . . . 

least as large as the previous integer in the list.) 16. Describe an algorithm for finding the smallest integer in 

a finite sequence of natural numbers. 

6. Describe an algorithm that takes as input a list of m in- 17. Describe an algorithm that locates the first occurrence of 
tegers and finds the number of negative integers in the the largest element in a finite list of integers, where the 
list. integers in the list are not necessarily distinct. 

7. Describe an algorithm that takes as input a list of n inte- 18. Describe an algorithm that locates the last occurrence of 


gers and finds the location of the last even integer in the 
list or returns 0 if there are no even integers in the list. 


the smallest element in a finite list of integers, where the 
integers in the list are not necessarily distinct. 


19. 


20 


21. 


22 


23. 


24 


25. 


26. 


27 


28 


Describe an algorithm that produces the maximum, me- 
dian, mean, and minimum of a set of three integers. (The 
median of a set of integers is the middle element in the list 
when these integers are listed in order of increasing size. 
The mean of a set of integers is the sum of the integers 
divided by the number of integers in the set.) 


Describe an algorithm for finding both the largest and the 
smallest integers in a finite sequence of integers. 


Describe an algorithm that puts the first three terms of 
a sequence of integers of arbitrary length in increasing 
order. 


Describe an algorithm to find the longest word in an En- 
glish sentence (where a sentence is a sequence of symbols, 
either a letter or a blank, which can then be broken into 
alternating words and blanks). 


Describe an algorithm that determines whether a function 
from a finite set of integers to another finite set of integers 
is onto. 


Describe an algorithm that determines whether a function 
from a finite set to another finite set is one-to-one. 


Describe an algorithm that will count the number of 1s in 
a bit string by examining each bit of the string to deter- 
mine whether it is a 1 bit. 


Change Algorithm 3 so that the binary search procedure 
compares x to d,, at each stage of the algorithm, with the 
algorithm terminating if x = a,,. What advantage does 
this version of the algorithm have? 


The ternary search algorithm locates an element ina list 
of increasing integers by successively splitting the list into 
three sublists of equal (or as close to equal as possible) 
size, and restricting the search to the appropriate piece. 
Specify the steps of this algorithm. 


Specify the steps of an algorithm that locates an element 
in a list of increasing integers by successively splitting 
the list into four sublists of equal (or as close to equal as 
possible) size, and restricting the search to the appropriate 
piece. 


In a list of elements the same element may appear several 
times. A mode of such a list is an element that occurs at least 
as often as each of the other elements; a list has more than 
one mode when more than one element appears the maximum 
number of times. 


29. 


30. 


31. 


32. 


Devise an algorithm that finds a mode in a list of nonde- 
creasing integers. (Recall that a list of integers is nonde- 
creasing if each term is at least as large as the preceding 
term.) 


Devise an algorithm that finds all modes. (Recall that a 
list of integers is nondecreasing if each term of the list is 
at least as large as the preceding term.) 


Devise an algorithm that finds the first term of a se- 
quence of integers that equals some previous term in the 
sequence. 


Devise an algorithm that finds all terms of a finite se- 
quence of integers that are greater than the sum of all 
previous terms of the sequence. 


33. 


34. 


35. 


36. 


* 37. 


38. 


39. 


40. 
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Devise an algorithm that finds the first term of a sequence 
of positive integers that is less than the immediately pre- 
ceding term of the sequence. 

Use the bubble sort to sort 6, 2, 3, 1, 5, 4, showing the 
lists obtained at each step. 

Use the bubble sort to sort 3, 1, 5, 7, 4, showing the lists 
obtained at each step. 

Use the bubble sort to sort d, f, k, m, a, b, showing the 
lists obtained at each step. 

Adapt the bubble sort algorithm so that it stops when 
no interchanges are required. Express this more efficient 
version of the algorithm in pseudocode. 

Use the insertion sort to sort the list in Exercise 34, show- 
ing the lists obtained at each step. 

Use the insertion sort to sort the list in Exercise 35, show- 
ing the lists obtained at each step. 

Use the insertion sort to sort the list in Exercise 36, show- 
ing the lists obtained at each step. 


The selection sort begins by finding the least element in the 
list. This element is moved to the front. Then the least element 


among the remaining elements is found and put into the sec- 
ond position. This procedure is repeated until the entire list 


has been sorted. 


41. 


44. 


45. 


46. 


Sort these lists using the selection sort. 
a) 3,5,4,1,2 b) 5,4, 3,2, 1 
ce) 1,2,3,4,5 


. Write the selection sort algorithm in pseudocode. 
. Describe an algorithm based on the linear search for de- 


termining the correct position in which to insert a new 
element in an already sorted list. 

Describe an algorithm based on the binary search for de- 
termining the correct position in which to insert a new 
element in an already sorted list. 

How many comparisons does the insertion sort use to sort 
the list 1,2,...,2? 


How many comparisons does the insertion sort use to sort 
the listn,n—1,...,2,1? 


The binary insertion sort is a variation of the insertion sort 
that uses a binary search technique (see Exercise 44) rather 
than a linear search technique to insert the ith element in the 
correct place among the previously sorted elements. 


47. 


48. 


* 49, 
50. 


Show all the steps used by the binary insertion sort to sort 
the list 3, 2, 4,5, 1, 6. 

Compare the number of comparisons used by the inser- 
tion sort and the binary insertion sort to sort the list 7, 4, 
3, 8, 1,5, 4, 2. 

Express the binary insertion sort in pseudocode. 

a) Devise a variation of the insertion sort that uses a lin- 
ear search technique that inserts the jth element in the 
correct place by first comparing it with the (j — 1)st 
element, then the (j — 2)th element if necessary, and 
so on. 

Use your algorithm to sort 3, 2, 4, 5, 1, 6. 

Answer Exercise 45 using this algorithm. 

Answer Exercise 46 using this algorithm. 


b) 
c) 
d) 
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51. When a list of elements is in close to the correct order, 
would it be better to use an insertion sort or its variation 
described in Exercise 50? 


52. Use the greedy algorithm to make change using quarters, 
dimes, nickels, and pennies for 
a) 87 cents. b) 49 cents. 
c) 99 cents. d) 33 cents. 

53. Use the greedy algorithm to make change using quarters, 
dimes, nickels, and pennies for 
a) 51 cents. b) 69 cents. 
c) 76 cents. d) 60 cents. 

54. Use the greedy algorithm to make change using quar- 


ters, dimes, and pennies (but no nickels) for each of the 
amounts given in Exercise 52. For which of these amounts 
does the greedy algorithm use the fewest coins of these 
denominations possible? 
55. Use the greedy algorithm to make change using quar- 
ters, dimes, and pennies (but no nickels) for each of the 
amounts given in Exercise 53. For which of these amounts 
does the greedy algorithm use the fewest coins of these 
denominations possible? 
Show that if there were a coin worth 12 cents, the greedy 
algorithm using quarters, 12-cent coins, dimes, nickels, 
and pennies would not always produce change using the 
fewest coins possible. 
Use Algorithm 7 to schedule the largest number of talks 
ina lecture hall from a proposed set of talks, if the starting 
and ending times of the talks are 9:00 a.m. and 9:45 A.M.; 
9:30 a.m. and 10:00 A.M.; 9:50 A.M. and 10:15 A.M.; 
10:00 a.m. and 10:30 a.m.; 10:10 a.m. and 10:25 a.M.; 
10:30 a.m. and 10:55 a.m.; 10:15 a.m. and 10:45 a.M.; 
10:30 a.m. and 11:00 a.m.; 10:45 a.m. and 11:30 a.M.; 
10:55 a.m. and 11:25 a.m.; 11:00 a.m. and 11:15 a.m. 
Show that a greedy algorithm that schedules talks in a lec- 
ture hall, as described in Example 7, by selecting at each 
step the talk that overlaps the fewest other talks, does not 
always produce an optimal schedule. 


56. 


57. 


58. 


*59. a) Devise a greedy algorithm that determines the fewest 
lecture halls needed to accommodate n talks given the 
starting and ending time for each talk. 

b) Prove that your algorithm is optimal. 

Suppose we have s men m,,m2,...,ms and s women 


W1, W2,..-, Ws. We wish to match each person with a member 


The Growth of Functions 


of the opposite gender. Furthermore, suppose that each person 
ranks, in order of preference, with no ties, the people of the 
opposite gender. We say that a matching of people of opposite 
genders to form couples is stable if we cannot find a man m 
and a woman w who are not assigned to each other such that 
m prefers w over his assigned partner and w prefers m to her 
assigned partner. 


60. Suppose we have three men m, mz, and m3 and three 
women wy, w2, and w3. Furthermore, suppose that the 
preference rankings of the men for the three women, from 
highest to lowest, are m1: w3, W1,W23 M2: W1, W2, W333: 
W2, W3, W1; and the preference rankings of the women for 
the three men, from highest to lowest, are wy: m , m2, 
m3; W2: M2, M1, M3; W3: m3, M2, my. For each of the 
six possible matchings of men and women to form three 
couples, determine whether this matching is stable. 

The deferred acceptance algorithm, also known as the Gale- 
Shapley algorithm, can be used to construct a stable matching 
of men and women. In this algorithm, members of one gender 
are the suitors and members of the other gender the suitees. 
The algorithm uses a sequence of rounds; in each round every 
suitor whose proposal was rejected in the previous round pro- 
poses to his or her highest ranking suitee who has not already 
rejected a proposal from this suitor. A suitee rejects all pro- 
posals except that from the suitor that this suitee ranks highest 
among all the suitors who have proposed to this suitee in this 
round or previous rounds. The proposal of this highest ranking 
suitor remains pending and is rejected in a later round if amore 
appealing suitor proposes in that round. The series of rounds 
ends when every suitor has exactly one pending proposal. All 
pending proposals are then accepted. 


61. 
62. 
* 63. 


Write the deferred acceptance algorithm in pseudocode. 
Show that the deferred acceptance algorithm terminates. 


Show that the deferred acceptance always terminates with 
a stable assignment. 


64. Show that the problem of determining whether a program 


with a given input ever prints the digit 1 is unsolvable. 


65. Show that the following problem is solvable. Given two 
programs with their inputs and the knowledge that exactly 


one of them halts, determine which halts. 


66. Show that the problem of deciding whether a specific 


program with a specific input halts is solvable. 


Introduction 


In Section 3.1 we discussed the concept of an algorithm. We introduced algorithms that solve a 
variety of problems, including searching for an element in a list and sorting a list. In Section 3.3 
we will study the number of operations used by these algorithms. In particular, we will estimate 
the number of comparisons used by the linear and binary search algorithms to find an element 
in a sequence of n elements. We will also estimate the number of comparisons used by the 
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Assessment 


Links 
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bubble sort and by the insertion sort to sort a list of n elements. The time required to solve a 
problem depends on more than only the number of operations it uses. The time also depends 
on the hardware and software used to run the program that implements the algorithm. However, 
when we change the hardware and software used to implement an algorithm, we can closely 
approximate the time required to solve a problem of size n by multiplying the previous time 
required by a constant. For example, on a supercomputer we might be able to solve a problem 
of size n a million times faster than we can on a PC. However, this factor of one million will 
not depend on n (except perhaps in some minor ways). One of the advantages of using big-O 
notation, which we introduce in this section, is that we can estimate the growth of a function 
without worrying about constant multipliers or smaller order terms. This means that, using big- 
O notation, we do not have to worry about the hardware and software used to implement an 
algorithm. Furthermore, using big-O notation, we can assume that the different operations used 
in an algorithm take the same time, which simplifies the analysis considerably. 

Big-O notation is used extensively to estimate the number of operations an algorithm uses 
as its input grows. With the help of this notation, we can determine whether it is practical to 
use a particular algorithm to solve a problem as the size of the input increases. Furthermore, 
using big-O notation, we can compare two algorithms to determine which is more efficient as 
the size of the input grows. For instance, if we have two algorithms for solving a problem, one 
using 100n? + 17n + 4 operations and the other using n? operations, big-O notation can help 
us see that the first algorithm uses far fewer operations when n is large, even though it uses more 
operations for small values of n, such as n = 10. 

This section introduces big-O notation and the related big-Omega and big-Theta notations. 
We will explain how big-O, big-Omega, and big-Theta estimates are constructed and establish 
estimates for some important functions that are used in the analysis of algorithms. 


Big-O Notation 


The growth of functions is often described using a special notation. Definition 1 describes this 
notation. 


Let f and g be functions from the set of integers or the set of real numbers to the set of real 
numbers. We say that f(x) is O(g(x)) if there are constants C and k such that 


If) = Cle@)| 


whenever x > k. [This is read as “ f(x) is big-oh of g(x).”] 


Remark: Intuitively, the definition that f(x) is O(g(x)) says that f(x) grows slower that some 
fixed multiple of g(x) as x grows without bound. 


The constants C and k in the definition of big-O notation are called witnesses to the 
relationship f(x) is O(g(x)). To establish that f(x) is O(g(x)) we need only one pair of 
witnesses to this relationship. That is, to show that f(x) is O(g(x)), we need find only one pair 
of constants C and k, the witnesses, such that | f(x)| < C|g(x)| whenever x > k. 

Note that when there is one pair of witnesses to the relationship f(x) is O(g(x)), there are 
infinitely many pairs of witnesses. To see this, note that if C and k are one pair of witnesses, 
then any pair C’ and k’, where C < C’ andk < k’, is also a pair of witnesses, because | f (x)| < 
Clg(x)| < C’|g(x)| whenever x > k’ > k. 
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THE HISTORY OF BIG-O NOTATION  Big-O notation has been used in mathematics for 
more than a century. In computer science it is widely used in the analysis of algorithms, as 
will be seen in Section 3.3. The German mathematician Paul Bachmann first introduced big-O 
notation in 1892 in an important book on number theory. The big-O symbol is sometimes called 
a Landau symbol after the German mathematician Edmund Landau, who used this notation 
throughout his work. The use of big-O notation in computer science was popularized by Donald 
Knuth, who also introduced the big-Q and big-© notations defined later in this section. 


WORKING WITHTHE DEFINITION OF BIG-O NOTATION A useful approach for find- 
ing a pair of witnesses is to first select a value of k for which the size of | f (x)| can be readily 
estimated when x > k and to see whether we can use this estimate to find a value of C for which 
| f (x)| < Clg(x)| for x > k. This approach is illustrated in Example 1. 


Show that f(x) = x? +2x + 1is O(x?). 


Solution: We observe that we can readily estimate the size of f(x) when x > 1 because x < x? 
and 1 < x* when x > 1. It follows that 


O<x7 42x41 < x7 42x? 4x7 = 4x? 


whenever x > 1, as shown in Figure 1. Consequently, we can take C = 4.andk = | as witnesses 
to show that f(x) is O(x). That is, f@= x? + 2x +1 < 4x? whenever x > 1. (Note that it 
is not necessary to use absolute values here because all functions in these equalities are positive 
when x is positive.) 

Alternatively, we can estimate the size of f(x) when x > 2. When x > 2, we have 2x < x 
and 1 < x2. Consequently, if x > 2, we have 


2 


O<x? +2x41< 2742? 4x7 = 3x’. 


It follows that C = 3 and k = 2 are also witnesses to the relation f(x) is O(x?). 


4x7) px? 42x41 x? 


The part of the graph of f(x) =x7+2x+1 
that satisfies f(x) < 4x? is shown in blue. 


x°+2x4+1< 4x7 forx> 1 


| | 
1 2 


FIGURE 1 The Function x? + 2x + 1 is O(x?). 
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Observe that in the relationship “f (x) is O(x7),” x? can be replaced by any function with 
larger values than x”. For example, f(x) is O(x3), f(x) is O(x? +x +7), and so on. 

It is also true that x? is O(x* + 2x + 1), because x” < x” + 2x + 1 whenever x > 1. This 
means that C = | and k = | are witnesses to the relationship x* is O(x? + 2x +1). < 


Note that in Example | we have two functions, f(x) = x? +2x +1 and g(x) = x”, such 
that f(x) is O(g(x)) and g(x) is O(f(x))—the latter fact following from the inequality 
x? <x?+42x +1, which holds for all nonnegative real numbers x. We say that two func- 
tions f(x) and g(x) that satisfy both of these big-O relationships are of the same order. We 


will return to this notion later in this section. 


Remark: The fact that f(x) is O(g(x)) is sometimes written f(x) = O(g(x)). However, the 
equals sign in this notation does not represent a genuine equality. Rather, this notation tells 
us that an inequality holds relating the values of the functions f and g for sufficiently large 
numbers in the domains of these functions. However, it is acceptable to write f(x) € O(g(x)) 
because O(g(x)) represents the set of functions that are O(g(x)). 


When f (x) is O(g(x)), and h(x) is a function that has larger absolute values than g(x) does 
for sufficiently large values of x, it follows that f(x) is O(h(x)). In other words, the function 
g(x) in the relationship f(x) is O(g(x)) can be replaced by a function with larger absolute 
values. To see this, note that if 


If@)|<Clg@)| ifx>k, 
and if |h(x)| > |g(x)| for all x > k, then 
\f@)(<ClA@)|. ifx>k. 


Hence, f(x) is O(h(x)). 

When big-O notation is used, the function g in the relationship f(x) is O(g(x)) is chosen 
to be as small as possible (sometimes from a set of reference functions, such as functions of the 
form x”, where n is a positive integer). 


PAUL GUSTAV HEINRICH BACHMANN (1837-1920) Paul Bachmann, the son of a Lutheran pastor, shared 
his father’s pious lifestyle and love of music. His mathematical talent was discovered by one of his teachers, even 
though he had difficulties with some of his early mathematical studies. After recuperating from tuberculosis 
in Switzerland, Bachmann studied mathematics, first at the University of Berlin and later at Géttingen, where 
he attended lectures presented by the famous number theorist Dirichlet. He received his doctorate under the 
German number theorist Kummer in 1862; his thesis was on group theory. Bachmann was a professor at Breslau 
and later at Miinster. After he retired from his professorship, he continued his mathematical writing, played the 
piano, and served as a music critic for newspapers. Bachmann’s mathematical writings include a five-volume 
survey of results and methods in number theory, a two-volume work on elementary number theory, a book on 


irrational numbers, and a book on the famous conjecture known as Fermat’s Last Theorem. He introduced big-O notation in his 
1892 book Analytische Zahlentheorie. 


Links 


EDMUND LANDAU (1877-1938) Edmund Landau, the son of a Berlin gynecologist, attended high school 
and university in Berlin. He received his doctorate in 1899, under the direction of Frobenius. Landau first taught 
at the University of Berlin and then moved to Gottingen, where he was a full professor until the Nazis forced 
him to stop teaching. Landau’s main contributions to mathematics were in the field of analytic number theory. 
In particular, he established several important results concerning the distribution of primes. He authored a 
three-volume exposition on number theory as well as other books on number theory and mathematical analysis. 
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Cg (x) 
FO) 
The part of the graph of f(x) that satisfies 


J (x) < Cg(x) is shown in color. 
g(x) 


f(x) < Cg(x) forx >k 


| 
k 


FIGURE 2. The Function f(x) is O(g(x)). 


In subsequent discussions, we will almost always deal with functions that take on only 
positive values. All references to absolute values can be dropped when working with big-O 
estimates for such functions. Figure 2 illustrates the relationship f(x) is O(g(x)). 

Example 2 illustrates how big-O notation is used to estimate the growth of functions. 


EXAMPLE 2 _ Show that 7x? is O(x3). 


Solution: Note that when x > 7, we have 7x? < x>. (We can obtain this inequality by multiplying 
both sides of x > 7 by x7.) Consequently, we can take C = 1 andk = 7 as witnesses to establish 


DONALD E. KNUTH (BORN 1938) Knuth grew up in Milwaukee, where his father taught bookkeeping at 
a Lutheran high school and owned a small printing business. He was an excellent student, earning academic 
achievement awards. He applied his intelligence in unconventional ways, winning a contest when he was in the 
eighth grade by finding over 4500 words that could be formed from the letters in “Ziegler’s Giant Bar.” This 
won a television set for his school and a candy bar for everyone in his class. 

Knuth had a difficult time choosing physics over music as his major at the Case Institute of Technology. He 
then switched from physics to mathematics, and in 1960 he received his bachelor of science degree, simultane- 
ously receiving a master of science degree by a special award of the faculty who considered his work outstanding. 
At Case, he managed the basketball team and applied his talents by constructing a formula for the value of each 
player. This novel approach was covered by Newsweek and by Walter Cronkite on the CBS television network. Knuth began graduate 
work at the California Institute of Technology in 1960 and received his Ph.D. there in 1963. During this time he worked as a consultant, 
writing compilers for different computers. 

Knuth joined the staff of the California Institute of Technology in 1963, where he remained until 1968, when he took a job as a 
full professor at Stanford University. He retired as Professor Emeritus in 1992 to concentrate on writing. He is especially interested 
in updating and completing new volumes of his series The Art of Computer Programming, a work that has had a profound influence 
on the development of computer science, which he began writing as a graduate student in 1962, focusing on compilers. In common 
jargon, “Knuth,” referring to The Art of Computer Programming, has come to mean the reference that answers all questions about 
such topics as data structures and algorithms. 

Knuth is the founder of the modern study of computational complexity. He has made fundamental contributions to the subject of 
compilers. His dissatisfaction with mathematics typography sparked him to invent the now widely used TeX and Metafont systems. 
TeX has become a standard language for computer typography. Two of the many awards Knuth has received are the 1974 Turing 
Award and the 1979 National Medal of Technology, awarded to him by President Carter. 

Knuth has written for a wide range of professional journals in computer science and in mathematics. However, his first 
publication, in 1957, when he was a college freshman, was a parody of the metric system called “The Potrzebie Systems of Weights 
and Measures,” which appeared in MAD Magazine and has been in reprint several times. He is a church organist, as his father was. 
He is also a composer of music for the organ. Knuth believes that writing computer programs can be an aesthetic experience, much 
like writing poetry or composing music. 

Knuth pays $2.56 for the first person to find each error in his books and $0.32 for significant suggestions. If you send him 
a letter with an error (you will need to use regular mail, because he has given up reading e-mail), he will eventually inform you 
whether you were the first person to tell him about this error. Be prepared for a long wait, because he receives an overwhelming 
amount of mail. (The author received a letter years after sending an error report to Knuth, noting that this report arrived several 
months after the first report of this error.) 
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the relationship Tx? is O(x3). Alternatively, when x > 1, we have 7x2 < 7x3, so that C =7 
and k = | are also witnesses to the relationship Tx? is O(x?). 


Example 3 illustrates how to show that a big-O relationship does not hold. 
Show that n? is not O(n). 


Solution: To show that n is not O(n), we must show that no pair of witnesses C and k exist 
such that n? < Cn whenever n > k. We will use a proof by contradiction to show this. 

Suppose that there are constants C and k for which n* < Cn whenever n > k. Observe that 
when n > 0 we can divide both sides of the inequality n? < Cn by n to obtain the equivalent 
inequality n < C. However, no matter what C and k are, the inequality n < C cannot hold for 
all n with n > k. In particular, once we set a value of k, we see that when n is larger than the 
maximum of k and C, it is not true that n < C even though n > k. This contradiction shows 
that n? in not O(n). 


Example 2 shows that 7x? is O(x>). Is it also true that x? is O(7x2)? 


Solution: To determine whether x? is O(7x2), we need to determine whether witnesses C and 
k exist, so that x? < C(7x*) whenever x > k. We will show that no such witnesses exist using 
a proof by contradiction. 

If C and k are witnesses, the inequality x < C(7x7) holds for all x > k. Observe that the 
inequality x? < C(7x7) is equivalent to the inequality x < 7C, which follows by dividing both 
sides by the positive quantity x”. However, no matter what C is, it is not the case that x < 7C 
for all x > k no matter what k is, because x can be made arbitrarily large. It follows that no 


witnesses C and k exist for this proposed big-O relationship. Hence, x? is not O(7x7). < 


Big-O Estimates for Some Important Functions 


Polynomials can often be used to estimate the growth of functions. Instead of analyzing the 
growth of polynomials each time they occur, we would like a result that can always be used to 
estimate the growth of a polynomial. Theorem | does this. It shows that the leading term of a 
polynomial dominates its growth by asserting that a polynomial of degree n or less is O(x”). 


Let f(x) = anx" + Gyoik 44-4 aye - ao, whete ap, a1, ....@,21, 4, ate reall num- 
bers. Then f(x) is O(x”). 


Proof: Using the triangle inequality (see Exercise 7 in Section 1.8), if x > 1 we have 


[f)| = |anx” + ap_1x""! +--+ + ayx + ao 

lan |x” + |dn—1|x"~! +--+ + lai|x + lao| 

= x" (lan + lan—al/x +--+ + lai|/x"-! + |ao|/x”) 
< x" (ldn| + |an—1] +++ + lai] + laol). 


IA 


This shows that 


[f(@)| < Cx", 
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EXAMPLE 5 


EXAMPLE 6 


where C = |ap| + |an—1| +--+ + |ao| whenever x > 1.Hence, the witnesses C = |an| + |an—1| 
+---++ |ao| and k = 1 show that f(x) is O(x”). 


We now give some examples involving functions that have the set of positive integers as 
their domains. 


How can big-O notation be used to estimate the sum of the first n positive integers? 


Solution: Because each of the integers in the sum of the first n positive integers does not exceed 
n, it follows that 


1+2+---+n<ntn+---+n=2’. 


From this inequality it follows that 1 + 2+3+---+n is O(n), taking C = 1 andk =1 as 
witnesses. (In this example the domains of the functions in the big-O relationship are the set of 
positive integers.) 


In Example 6 big-O estimates will be developed for the factorial function and its loga- 
rithm. These estimates will be important in the analysis of the number of steps used in sorting 
procedures. 


Give big-O estimates for the factorial function and the logarithm of the factorial function, where 
the factorial function f(n) = n! is defined by 


ni=1-2-3--+--n 
whenever n is a positive integer, and 0! = 1. For example, 

W=1, 2!=1-2=2, 3!=1-2-3=6, 44=1-2-3-4=24. 
Note that the function n! grows rapidly. For instance, 


20! = 2,432,902,008,176,640,000. 


Solution: A big-O estimate for n! can be obtained by noting that each term in the product does 
not exceed n. Hence, 


nl =1-2-3. +00 -n 
<n-n-n-: see HW 
=n", 


This inequality shows that n! is O(n"), taking C = | andk = | as witnesses. Taking logarithms 
of both sides of the inequality established for n!, we obtain 


logn! < logn” =nlogn. 


This implies that log n! is O(n logn), again taking C = | and k = | as witnesses. < 


EXAMPLE 7 


3.2 The Growth of Functions 211 


In Section 4.1, we will show that n < 2” whenever n is a positive integer. Show that this 
inequality implies that n is O(2”), and use this inequality to show that logn is O(n). 


Solution: Using the inequality n < 2”, we quickly can conclude that n is O(2”) by taking k = 
C = | as witnesses. Note that because the logarithm function is increasing, taking logarithms 
(base 2) of both sides of this inequality shows that 


logn <n. 
It follows that 
logn is O(n). 


(Again we take C = k = | as witnesses.) 
If we have logarithms to a base b, where b is different from 2, we still have log, n is O(n) 
because 


logn n 


it — 
O85" logb * logb 

whenever n is a positive integer. We take C = 1/logb and k = 1 as witnesses. (We have used 
Theorem 3 in Appendix 2 to see that log, n = logn/ log b.) 


As mentioned before, big-O notation is used to estimate the number of operations needed to 
solve a problem using a specified procedure or algorithm. The functions used in these estimates 
often include the following: 


1, logn, n, nlogn, n?, 2”, n! 


Using calculus it can be shown that each function in the list is smaller than the succeeding 
function, in the sense that the ratio of a function and the succeeding function tends to zero 
as n grows without bound. Figure 3 displays the graphs of these functions, using a scale for 
the values of the functions that doubles for each successive marking on the graph. That is, the 
vertical scale in this graph is logarithmic. 


4096 
2048 
1024 


FIGURE 3 A Display of the Growth of Functions Commonly Used in Big-O Estimates. 
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USEFUL BIG-O ESTIMATES INVOLVING LOGARITHMS, POWERS, AND EXPONEN- 
TIAL FUNCTIONS We now give some useful facts that help us determine whether big-O 
relationships hold between pairs of functions when each of the functions is a power of a loga- 
rithm, a power, or an exponential function of the form b” where b > 1. Their proofs are left as 
Exercises 57-60 for readers skilled with calculus. 

Theorem | shows that if f(n) is a polynomial of degree d, then f(n) is O(n“). Applying 
this theorem, we see that if d > c > 1, then n° is O(nt ). We leave it to the reader to show 
that the reverse of this relationship does not hold. Putting these facts together, we see that if 
d>c> 1,then 


n° is O(n“), but n@ is not O(n‘). 


In Example 7 we showed that log, n is O(n) whenever b > 1. More generally, whenever b > 1 
and c and d are positive, we have 


(log, n)° is O(n“), but n@ is not (O (log, n)°). 


This tells us that every positive power of the logarithm of n to the base b, where b > 1, is big-O 
of every positive power of n, but the reverse relationship never holds. 

In Example 7, we also showed that n is O(2”). More generally, whenever d is positive and 
b > 1, we have 


n@ is O(b"), but b” is not O(n”). 


This tells us that every power of n is big-O of every exponential function of n with a base 
that is greater than one, but the reverse relationship never holds. Furthermore, we have when 
c> b> 1, 


b" is O(c") but c” is not O(b"). 


This tells us that if we have two exponential functions with different bases greater than one, one 
of these functions is big-O of the other if and only if its base is smaller or equal. 


The Growth of Combinations of Functions 


Many algorithms are made up of two or more separate subprocedures. The number of steps 
used by a computer to solve a problem with input of a specified size using such an algorithm is 
the sum of the number of steps used by these subprocedures. To give a big-O estimate for the 
number of steps needed, it is necessary to find big-O estimates for the number of steps used by 
each subprocedure and then combine these estimates. 

Big-O estimates of combinations of functions can be provided if care is taken when different 
big-O estimates are combined. In particular, it is often necessary to estimate the growth of the 
sum and the product of two functions. What can be said if big-O estimates for each of two 
functions are known? To see what sort of estimates hold for the sum and the product of two 
functions, suppose that f;(x) is O(g;(x)) and f(x) is O(g2(x)). 

From the definition of big-O notation, there are constants C,, C2, k,, and k2 such that 


Ifi®)| < Cilgi@)| 


when x > k, and 


| fa(x)| < C2lg2(x)| 


THEOREM 2 


COROLLARY 1 


THEOREM 3 
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when x > k. To estimate the sum of fj (x) and f(x), note that 


IA + fC) = 1A) + A@)| 
< |fitv)| + |fo@)| — using the triangle inequality Ja + b| < |a| + ||. 


When x is greater than both k; and kg, it follows from the inequalities for | f;(x)| and | fo(x)| 
that 


I fi@)| + | f2)1 S Cilgi@&)| + Colg2@)| 
< Cilg(x)| + Calg@)| 
= (C} + C2)|g(x)| 
= Clg(x)I, 
where C = C; + Cy and g(x) = max(|g1(x)|, |g2(x)|). [Here max(a, b) denotes the maximum, 
or larger, of a and b.] 


This inequality shows that |(f1 + fo)(x)| < Clg(x)| whenever x >k, where k= 
max(k, k2). We state this useful result as Theorem 2. 


Suppose that f;(x) is O(g ;(x)) and that fo(x) 1s O(g2(x)). Then (fj + f2)(x) is 
O(max(|g1(x)|, |g2(x)|)). 


We often have big-O estimates for f; and f> in terms of the same function g. In this situation, 
Theorem 2 can be used to show that (f| + f2)(x) is also O(g(x)), because max(g(x), g(x)) = 
g(x). This result is stated in Corollary 1. 


Suppose that f;(x) and fo(x) are both O(g(x)). Then (f; + f2)(x) is O(g(x)). 


In a similar way big-O estimates can be derived for the product of the functions f; and fo. 
When x is greater than max(k1, k2) it follows that 


(Ai f2@)| = AOI F201 
< Cilgi(x)|C2|g2(%)| 
< C1C2|(g1g2)(x)| 
< C\(g1g2)(x)I, 
where C = C,C2. From this inequality, it follows that f\(x) fo(x) is O(g1g2(x)), because 


there are constants C and k, namely, C = C,C2 and k = max(ky, k2), such that |(f) f2)(x)| < 
C|g1(x)g2(x)| whenever x > k. This result is stated in Theorem 3. 


Suppose that f(x) is O(g1(x)) and fy(x) is O(g2(x)). Then (fi f2)(x) is O(g1 (4) g2(x)). 


The goal in using big- O notation to estimate functions is to choose a function g(x) as simple 
as possible, that grows relatively slowly so that f(x) is O(g(x)). Examples 8 and 9 illustrate 
how to use Theorems 2 and 3 to do this. The type of analysis given in these examples is often 
used in the analysis of the time used to solve problems using computer programs. 
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EXAMPLE 8 


EXAMPLE 9 


Q and © are the Greek 
uppercase letters omega 
and theta, respectively. 


DEFINITION 2 


EXAMPLE 10 


Give a big-O estimate for f(n) = 3n log(n!) + (n? + 3) logn, where n is a positive integer. 


Solution: First, the product 3n log(n!) will be estimated. From Example 6 we know that log(n!) 
is O(n logn). Using this estimate and the fact that 3n is O(n), Theorem 3 gives the estimate 
that 3n log(n!) is O(n? logn). 

Next, the product (n? +3) log n will be estimated. Because (n? +3) < 2n? whenn > 2, it 
follows that n? + 3 is O(n”). Thus, from Theorem 3 it follows that (n? + 3) lognis O (n2 logn). 
Using Theorem 2 to combine the two big-O estimates for the products shows that f(n) = 
3n log(n!) + (n? + 3) logn is O(n? logn). 


Give a big-O estimate for f(x) = (x + 1) log(x? +1) + 3x?. 


Solution: First, a big-O estimate for (x + 1) log(x? +1) will be found. Note that (x + 1) is 
O(x). Furthermore, x2 + 1 < 2x? when x > 1. Hence, 


log(x? +1) < log(2x7) =log2+ log x? = log2+2logx < 3logx, 


if x > 2. This shows that log (x? + 1) is O(log x). 

From Theorem 3 it follows that (x + 1) log(x? + 1) is O(x log x). Because 3x2 is O(x”), 
Theorem 2 tells us that f(x) is O(max(x log x, x”)). Because x logx < x’, forx > 1, it follows 
that f (x) is O(x?). 4 


Big-Omega and Big-Theta Notation 


Big-O notation is used extensively to describe the growth of functions, but it has limitations. In 
particular, when f (x) is O(g(x)), we have an upper bound, in terms of g(x), for the size of f (x) 
for large values of x. However, big-O notation does not provide a lower bound for the size of f (x) 
for large x. For this, we use big-Omega (big-(2) notation. When we want to give both an upper 
and a lower bound on the size of a function f (x), relative to areference function g(x), we use big- 
Theta (big-©) notation. Both big-Omega and big-Theta notation were introduced by Donald 
Knuth in the 1970s. His motivation for introducing these notations was the common misuse of 
big-O notation when both an upper and a lower bound on the size of a function are needed. 

We now define big-Omega notation and illustrate its use. After doing so, we will do the 
same for big-Theta notation. 


Let f and g be functions from the set of integers or the set of real numbers to the set of real 
numbers. We say that f(x) is Q(g(x)) if there are positive constants C and k such that 


If@)| 2 Cle@)| 


whenever x > k. [This is read as “ f(x) is big-Omega of g(x).”] 


There is a strong connection between big-O and big-Omega notation. In particular, f(x) is 
Q(g(x)) if and only if g(x) is O(f (x)). We leave the verification of this fact as a straightforward 
exercise for the reader. 


The function f(x) = 8x3 + 5x2 +7 is Q(g(x)), where g(x) is the function g(x) = x>. This 
is easy to see because f(x) = 8x° + 5x + 7 > 8x? for all positive real numbers x. This is 
equivalent to saying that g(x) = x? is O(8x* + 5x? + 7), which can be established directly by 
turning the inequality around. < 


DEFINITION 3 


EXAMPLE 11 


Extra 
Examples 


EXAMPLE 12 


< 
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Often, it is important to know the order of growth of a function in terms of some relatively 
simple reference function such as x” when n is a positive integer or c*, where c > 1. Knowing 
the order of growth requires that we have both an upper bound and a lower bound for the size of 
the function. That is, given a function f(x), we want a reference function g(x) such that f (x) 
is O(g(x)) and f (x) is Q(g(x)). Big-Theta notation, defined as follows, is used to express both 
of these relationships, providing both an upper and a lower bound on the size of a function. 


Let f and g be functions from the set of integers or the set of real numbers to the set of real 
numbers. We say that f(x) is O(g(x)) if f(x) is O(g(x)) and f(x) is Q(g(x)). When f(x) 
is O(g(x)) we say that f is big-Theta of g(x), that f(x) is of order g(x), and that f(x) and 
g(x) are of the same order. 


When f(x) is O(g(x)), it is also the case that g(x) is O( f (x)). Also note that f(x) is O(g(x)) 
if and only if f(x) is O(g(x)) and g(x) is O(f (x)) (see Exercise 31). Furthermore, note that 
Ff (x) is ©(g(x)) if and only if there are real numbers C; and C2 and a positive real number k 
such that 


Cilg@)| < lf @)| < Cale@)| 


whenever x > k. The existence of the constants C;, C2, and k tells us that f(x) is Q(g(x)) and 
that f(x) is O(g(x)), respectively. 

Usually, when big-Theta notation is used, the function g(x) in O(g(x)) is a relatively simple 
reference function, such as x”, c*, log x, and so on, while f(x) can be relatively complicated. 


We showed (in Example 5) that the sum of the first n positive integers is O(n). Is this sum of 
order n?? 


Solution: Let f(n) = 1+2+3+----+n. Because we already know that f(n) is O(n), to 
show that f (7) is of order n2 we need to find a positive constant C such that f(n) > C n2 for 
sufficiently large integers n. To obtain a lower bound for this sum, we can ignore the first half 
of the terms. Summing only the terms greater than [n/2], we find that 


[2 een 2 [n/2| eC [ay2) 4 1) eee en 
> [n/2] + [n/2] +---+ fn/2] 
= (n — [n/2] + 1) [n/2] 
> (n/2)(n/2) 
=n? /4. 


This shows that f (7) is Q(n2). We conclude that f (n) is of order n’, or in symbols, f(n) is 
@(n’). < 


Show that 3x? + 8x log x is O(x?). 
Solution: Because 0 < 8x logx < 8x2, it follows that 3x? + 8x logx < llx? for x > 1. 


Consequently, 3x7 + 8x logx is O(x*). Clearly, x? is O(3x? + 8x log x). Consequently, 
3x? + 8x log x is Q(x”). > | 
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One useful fact is that the leading term of a polynomial determines its order. For example, 
if fx~= 3x5 + x4 + 17x3 4+ 2, then Ff (x) is of order x>. This is stated in Theorem 4, whose 
proof is left as Exercise 50. 


. , dy are real numbers with 


The polynomials 3x8 + 10x? + 221x? + 1444, x!9 — 18x4 — 10,112, and —x99 4 40,001x?8 
+ 100,003x are of orders x8, x!9, and x%, respectively. < 


Unfortunately, as Knuth observed, big-O notation is often used by careless writers and 
speakers as if it had the same meaning as big-Theta notation. Keep this in mind when you see 
big-O notation used. The recent trend has been to use big-Theta notation whenever both upper 


THEOREM 4 Ibi (C2) = Grad’ 46 Qn_1x" | +-++++ a,x + a0, where ao, aj,.. 
ay, # 0. Then f(x) is of order x”. 
EXAMPLE 13 
and lower bounds on the size of a function are needed. 
Exercises 


In Exercises 1—14, to establish a big-O relationship, find wit- 
nesses C and k such that | f(x)| < C|g(x)| whenever x > k. 


1. 


Nn 


10. 
11. 


Determine whether each of these functions is O(x). 


a) f(x) =17x + 11 
ce) f(x) =xlogx 
e) f(x) =2* 


b) f(x) =x? + 1000 
d) f(x) =x4/2 
f) f@) = Lx]: [x] 


. Use the definition of “f(x) is O(g(x))” to show that 


x4 49x3 + 4x +7 is O(x‘). 


. Use the definition of “f(x) is O(g(x))” to show that 


2* + 17 is O(3*). 


. Show that (x? + 1)/(x + 1) is O(x). 
. Show that (x3 + 2x)/(2x + 1) is O(x?). 
. Find the least integer n such that f(x) is O(x”) for each 


of these functions. 

a) f(x) = 2x3 4+ x? log x 

b) f(x) = 3x? + (log x)* 

oe) f(x) = @4t +274 D/03 +1) 
d) f(x) = (x* + 5logx)/(x* + 1) 


. Find the least integer n such that f(x) is O(x”) for each 


of these functions. 

a) f(x) = 2x74 x7 logx 

b) f(x) = 3x° + (log x)4 

co) f(x) = @t +274 D/O44+ 1) 
d) f(x) = («3 + 5logx)/(at + 1) 


. Show that x2-+4x +17 is O(x>) but that x? is not 


O(x? + 4x +17). 
Show that x? is O(x*) but that x+ is not O(x3). 
Show that 3x4 + 1 is O(x4/2) and x4/2 is O3x4 + 1). 


15. 
16. 
17. 


18. 


19. 


20. 


21. 


22. 


23. 


24, 


. Show that x log x is O(x?) but that x? is not O(x log x). 
13. 


Show that 2” is O(3”) but that 3” is not O(2”). (Note that 
this is a special case of Exercise 60.) 


14. Determine whether x? is O(g(x)) for each of these func- 
a) f(x) = 10 b) f(x) =3x+7 tions g(x). 
oc) f(x)=x?7+x4+1 — d) f(x) =Slogx ana b) eG =x? 
e) fx)=L] f) f(x) = [x/2] ce) gx)= x2 4x3 d) g(x) = x2 4x4 
. Determine whether each of these functions is O (x2). e) g(x) =3* f) g(x) =x3/2 


Explain what it means for a function to be O(1). 

Show that if f(x) is O(x), then f(x) is O(x?). 
Suppose that f(x), g(x), and /(x) are functions such that 
f(x) is O(g(x)) and g(x) is O(A(x)). Show that f (x) is 
O(h(x)). 

Let k be a positive integer. Show that 1* + 24 +..-4+ né* 
is O(n*+!), 

Determine whether each of the functions 2”+! and 2” is 
O(2"). 

Determine whether each of the functions log(m + 1) and 
log(n? + 1) is O(logn). 

Arrange the functions ./n, 1000 log n,n log n, 2n!, 2", 3”, 
and n? /1,000,000 in a list so that each function is big-O 
of the next function. 

Arrange the function (1.5)",n 100 | (log ny, J/nlogn, 10", 
(n!)?, andn?? + n°8 ina list so that each function is big-O 
of the next function. 

Suppose that you have two different algorithms for solv- 
ing a problem. To solve a problem of size n, the first 
algorithm uses exactly n(logn) operations and the sec- 
ond algorithm uses exactly n3/? operations. As n grows, 
which algorithm uses fewer operations? 

Suppose that you have two different algorithms for solv- 
ing a problem. To solve a problem of size n, the first 
algorithm uses exactly n?2” operations and the second 
algorithm uses exactly n! operations. As n grows, which 
algorithm uses fewer operations? 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 
37. 
38. 


Give as good a big-O estimate as possible for each of 
these functions. 


a) (n?+8)(n+ 1) b) (nlogn +.n7)(n3 + 2) 

c) (n! + 2")(n3 + log(n? + 1)) 

Give a big- O estimate for each of these functions. For the 

function g in your estimate f(x) is O(g(x)), use a simple 

function g of smallest order. 

a) (n3+n? logn)(logn+1) + (17 logn+19)(n?+2) 

b) (2 +n7)(n? + 3”) 

ce) (n” +n2” +5")(n! + 5”) 

Give a big- O estimate for each of these functions. For the 

function g in your estimate that f(x) is O(g(x)), use a 

simple function g of the smallest order. 

a) nlog(n? + 1) +n? logn 

b) ilogn+ 12+ dogn + 1)(n? + 1) 

On n2 

c) n~ +n 

For each function in Exercise 1, determine whether that 

function is Q(x) and whether it is O(x). 

For each function in Exercise 2, determine whether that 

function is Q(x?) and whether it is @(x2). 

Show that each of these pairs of functions are of the same 

order. 

a) 3x+7,x 

b) 2x2 +x —7, x? 

o) [x +1/2],x 

d) log(x? + 1), log, x 

e) logig x, logs x 

Show that f(x) is O(g(x)) if and only if f(x) is O(g(x)) 

and g(x) is O(f (x)). 

Show that if f(x) and g(x) are functions from the set 

of real numbers to the set of real numbers, then f(x) is 

O(g(x)) if and only if g(x) is Q(f(x)). 

Show that if f(x) and g(x) are functions from the set 

of real numbers to the set of real numbers, then f(x) is 

©(g(x)) if and only if there are positive constants k, C1, 

and C2 such that Ci|g(x)| < |f(x)| < Colg(x)| when- 

ever x >k. 

a) Show that 3x” + x + 1 is @(3x7) by directly finding 
the constants k, C,, and C2 in Exercise 33. 

b) Express the relationship in part (a) using a picture 
showing the functions 8x2 txt 1, C, - 3x2, and 
C2 - 3x2, and the constant k on the x-axis, where 
C1, C2, and k are the constants you found in part (a) 
to show that 3x2 + x + 1 is @(3x?). 

Express the relationship f (x) is O(g(x)) using a picture. 

Show the graphs of the functions f(x), Ci|g(x)|, and 

C2|g(x)|, as well as the constant k on the x-axis. 

Explain what it means for a function to be Q(1). 

Explain what it means for a function to be @(1). 


Give a big-O estimate of the product of the first n odd 
positive integers. 


39. 


40. 


41. 


42. 


43. 


44, 


45. 


46. 


47. 


48. 


49. 


50. 
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Show that if f and g are real-valued functions such that 
f (x) is O(g(x)), then for every positive integer n, f” (x) 
is O(g”(x)). [Note that f"(x) = f(x)”.] 

Show that for all real numbers a and b with a > 1 and 
b> 1, if f(x) is O(log, x), then f(x) is O(log, x). 
Suppose that f(x) is O(g(x)) where f and g are in- 
creasing and unbounded functions. Show that log | f (x)| 
is O(log |g(x)|). 

Suppose that f(x) is O(g(x)). Does it follow that 2/) 
is O(28))? 

Let fi (x) and f2(x) be functions from the set of real 
numbers to the set of positive real numbers. Show that if 
fi (x) and f(x) are both ©(g(x)), where g(x) is a func- 
tion from the set of real numbers to the set of positive real 
numbers, then f(x) + f2(x) is O(g(x)). Is this still true 
if fi (x) and f2(x) can take negative values? 

Suppose that f(x), g(x), and h(x) are functions such that 
f (x) is O(g(x)) and g(x) is O(h(x)). Show that f(x) is 
O(h(@)). 

If fi(x) and f2(x) are functions from the set of positive 
integers to the set of positive real numbers and f(x) and 
f2(x) are both O(g(x)), is (fi — f2)(&) also O(g(x))? 
Either prove that it is or give a counterexample. 

Show that if f(x) and f(x) are functions from the set 
of positive integers to the set of real numbers and f(x) 
is O(gi(x)) and f2(x) is O(g2(x)), then (fi f2)(x) is 
O((g1g2)(x)). 

Find functions f and g from the set of positive integers 
to the set of real numbers such that f(n) is not O(g(n)) 
and g(n) is not O(f(n)). 

Express the relationship f (x) is Q(g(x)) using a picture. 
Show the graphs of the functions f (x) and Cg(x), as well 
as the constant k on the real axis. 

Show that if f(x) is O(gi(x)), fo(x) is O(g2(x)), and 
f2(x) A Oand go(x) ¥ O forall real numbers x > 0, then 
(fi/f2)(x) is O((g1/g2)(x)). 

Show that if f(x) = a@yx” + dy—yx"~!4+++++ayx + 
ao, where ao, a1, ..., @,—1, and dy are real numbers and 
an # 0, then f(x) is O(x”). 


Big-O, big-Theta, and big-Omega notation can be extended 
to functions in more than one variable. For example, the state- 
ment f(x, y) is O(g(x, y)) means that there exist constants C, 
ky, and kg such that | f(x, y)| < Clge(x, y)| whenever x > ky 
and y > ko. 


51. 
52. 
53. 
54. 
55. 
56. 
57. 


58. 


Define the statement f(x, y) is O(g(x, y)). 

Define the statement f(x, y) is Q(g(x, y)). 

Show that (x? + xy +x log y)? is O(x®y3). 

Show that x>y? + x4y4 + x3y> is Q(x y3). 

Show that Lxy] is O(xy). 

Show that [xy] is Q(xy). 

(Requires calculus) Show that if c > d > 0, then n?@ is 
O(n‘), but n° is not O(n”). 

(Requires calculus) Show that if b> 1 and c and d 
are positive, then (log, 7n)° is O(n), but n@ is not 
O((log, 1)°). 
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59. (Requires calculus) Show that if d is positive and b > 1, 
then n@ is O(b") but b” is not O(n*). 

60. (Requires calculus) Show that if c > b > 1, then b” is 
O(c") but c” is not O(b"). 

The following problems deal with another type of asymptotic 

notation, called little-o notation. Because little-o notation is 

based on the concept of limits, a knowledge of calculus is 

needed for these problems. We say that f (x) is o(g(x)) [read 

Ff (x) is “little-oh” of g(x)], when 


tim S <5 


X>0O Q(X) — 


61. (Requires calculus) Show that 
a) x? is o(x?). b) x logx is o(x?). 

c) x? is 0(2*). d) x? +x + Lisnoto(x?). 

62. (Requires calculus) 

a) Show that if f(x) and g(x) are functions such that 
f(x) is o(g(x)) and c is a constant, then cf (x) is 
o(g(x)), where (cf) (x) = cf (x). 

b) Show that if f(x), fo(x), and g(x) are functions 
such that fi(x) is o(g(x)) and fo(x) is o(g(x)), 
then (fi + f2)(x) is o(g(x)), where (fi + f2)(x) = 
fi) + f2(x). 

63. (Requires calculus) Represent pictorially that x log x is 
o(x”) by graphing x logx, x”, and x logx/x”. Explain 
how this picture shows that x log x is o(x?). 

64. (Requires calculus) Express the relationship f(x) is 
o(g(x)) using a picture. Show the graphs of f(x), g(x), 
and f (x)/g(x). 

*65. (Requires calculus) Suppose that f(x) is o(g(x)). Does 

it follow that 2/@) is 0(28%)? 

* 66. (Requires calculus) Suppose that f(x) is o(g(x)). Does 
it follow that log | f (x)| is o(log |g(x)|)? 

67. (Requires calculus) The two parts of this exercise describe 
the relationship between little-o and big-O notation. 

a) Show that if f(x) and g(x) are functions such that 
f(x) is o(g(x)), then f(x) is O(g(x)). 

b) Show that if f(x) and g(x) are functions such that 
f(x) is O(g(x)), then it does not necessarily follow 
that f(x) is o(g(x)). 

68. (Requires calculus) Show that if f (x) is a polynomial of 
degree n and g(x) is a polynomial of degree m where 
m > n, then f(x) is o(g(x)). 
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69. (Requires calculus) Show that if fj (x) is O(g(x)) and 
fr(x) is o(g(x)), then fix) + fo(x) is O(g(x)). 
70. (Requires calculus) Let H, be the nth harmonic number 


ae ee eee ie 
aa 2 3 n 


Show that H, is O(logn). [Hint: First establish the in- 
equality 


el ey 
Dio<f sax 
1 x 


j=at 


by showing that the sum of the areas of the rectangles of 
height 1/7 with base from j — 1 to j, for j = 2,3,...,n, 
is less than the area under the curve y = 1/x from 2 ton.] 


*71. Show that n logn is O(logn!). 


t=" 72, Determine whether log n! is @(n logn). Justify your an- 


swer. 
*73. Show that log n! is greater than (nlogn)/4 for 
n>4. [Hint: Begin with the inequality n!> 
n(n — 1)(n — 2)--- [n/2].] 
Let f(x) and g(x) be functions from the set of real num- 
bers to the set of real numbers. We say that the func- 
tions f and g are asymptotic and write f(x) ~ g(x) 
if limy—+ 00 fx)/g(x) =1. 
74. (Requires calculus) For each of these pairs of functions, 
determine whether f and g are asymptotic. 
a) f(x) = x? + 3x +7, g(x) =x7410 
b) f(x) =x? log x, g(x) = x3 
c) f(x) = x4 +1logBx® +7), 
g(x) = (x2 + 17x +3)? 
d) f(x) =O34+x27 4x4 1%, 
g(x) = xtt+x3 4x27 4x41). 
75. (Requires calculus) For each of these pairs of functions, 
determine whether f and g are asymptotic. 
a) f(x) = log(x* + 1), g(x) = logx 
b) f(x) = 2", g(x) = 2"47 
oS) 2a =e 
VfoS) "26S 


Introduction 


When does an algorithm provide a satisfactory solution to a problem? First, it must always 
produce the correct answer. How this can be demonstrated will be discussed in Chapter 5. 
Second, it should be efficient. The efficiency of algorithms will be discussed in this section. 
How can the efficiency of an algorithm be analyzed? One measure of efficiency is the time 
used by a computer to solve a problem using the algorithm, when input values are of a specified 


EXAMPLE 1 


Extra p> 
Examples 2 
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size. A second measure is the amount of computer memory required to implement the algorithm 
when input values are of a specified size. 

Questions such as these involve the computational complexity of the algorithm. An analysis 
of the time required to solve a problem of a particular size involves the time complexity of the 
algorithm. An analysis of the computer memory required involves the space complexity of 
the algorithm. Considerations of the time and space complexity of an algorithm are essential 
when algorithms are implemented. It is obviously important to know whether an algorithm will 
produce an answer in a microsecond, a minute, or a billion years. Likewise, the required memory 
must be available to solve a problem, so that space complexity must be taken into account. 

Considerations of space complexity are tied in with the particular data structures used to 
implement the algorithm. Because data structures are not dealt with in detail in this book, space 
complexity will not be considered. We will restrict our attention to time complexity. 


Time Complexity 


The time complexity of an algorithm can be expressed in terms of the number of operations 
used by the algorithm when the input has a particular size. The operations used to measure time 
complexity can be the comparison of integers, the addition of integers, the multiplication of 
integers, the division of integers, or any other basic operation. 

Time complexity is described in terms of the number of operations required instead of actual 
computer time because of the difference in time needed for different computers to perform basic 
operations. Moreover, it is quite complicated to break all operations down to the basic bit oper- 
ations that a computer uses. Furthermore, the fastest computers in existence can perform basic 
bit operations (for instance, adding, multiplying, comparing, or exchanging two bits) in 107!! 
second (10 picoseconds), but personal computers may require 10~® second (10 nanoseconds), 
which is 1000 times as long, to do the same operations. 

We illustrate how to analyze the time complexity of an algorithm by considering Algorithm | 
of Section 3.1, which finds the maximum of a finite set of integers. 


Describe the time complexity of Algorithm | of Section 3.1 for finding the maximum element 
in a finite set of integers. 


Solution: The number of comparisons will be used as the measure of the time complexity of the 
algorithm, because comparisons are the basic operations used. 

To find the maximum element of a set with n elements, listed in an arbitrary order, the 
temporary maximum is first set equal to the initial term in the list. Then, after a comparison 
i < nhas been done to determine that the end of the list has not yet been reached, the temporary 
maximum and second term are compared, updating the temporary maximum to the value of 
the second term if it is larger. This procedure is continued, using two additional comparisons 
for each term of the list—one i < n, to determine that the end of the list has not been reached 
and another max < a;, to determine whether to update the temporary maximum. Because two 
comparisons are used for each of the second through the nth elements and one more comparison 
is used to exit the loop when i = n + 1, exactly 2(n — 1) + 1 = 2n — 1 comparisons are used 
whenever this algorithm is applied. Hence, the algorithm for finding the maximum of a set 
of n elements has time complexity ©(7), measured in terms of the number of comparisons 
used. Note that for this algorithm the number of comparisons is independent of particular input 
of n numbers. 


Next, we will analyze the time complexity of searching algorithms. 
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EXAMPLE 2 


EXAMPLE 3 


Describe the time complexity of the linear search algorithm (specified as Algortihm 2 in 
Section 3.1). 


Solution: The number of comparisons used by Algorithm 2 in Section 3.1 will be taken as the 
measure of the time complexity. At each step of the loop in the algorithm, two comparisons 
are performed—one i < n, to see whether the end of the list has been reached and one x < aj, 
to compare the element x with a term of the list. Finally, one more comparison i < n is made 
outside the loop. Consequently, if x = a;, 2i + 1 comparisons are used. The most comparisons, 
2n + 2, are required when the element is not in the list. In this case, 2n comparisons are used 
to determine that x is not aj, fori = 1,2,...,, an additional comparison is used to exit the 
loop, and one comparison is made outside the loop. So when x is not in the list, a total of 2n + 2 
comparisons are used. Hence, a linear search requires ©(n) comparisons in the worst case, 
because 2n + 2 is O(n). < 


WORST-CASE COMPLEXITY The type of complexity analysis done in Example 2 is a worst- 
case analysis. By the worst-case performance of an algorithm, we mean the largest number of 
operations needed to solve the given problem using this algorithm on input of specified size. 
Worst-case analysis tells us how many operations an algorithm requires to guarantee that it will 
produce a solution. 


Describe the time complexity of the binary search algorithm (specified as Algorithm 3 in 
Section 3.1) in terms of the number of comparisons used (and ignoring the time required to 
compute m = |(i + j)/2] in each iteration of the loop in the algorithm). 


Solution: For simplicity, assume there are n = 2* elements in the list aj, d2,..., dn, Wherekisa 
nonnegative integer. Note that k = logn. (If n, the number of elements in the list, is not a power 
of 2, the list can be considered part of a larger list with 2+! elements, where 2 <n < 2*+!, 
Here 2‘*! is the smallest power of 2 larger than n.) 

At each stage of the algorithm, i and /, the locations of the first term and the last term of 
the restricted list at that stage, are compared to see whether the restricted list has more than one 
term. If i < j, a comparison is done to determine whether x is greater than the middle term of 
the restricted list. 

At the first stage the search is restricted to a list with 2‘~! terms. So far, two comparisons 
have been used. This procedure is continued, using two comparisons at each stage to restrict 
the search to a list with half as many terms. In other words, two comparisons are used at the 
first stage of the algorithm when the list has 2 elements, two more when the search has been 
reduced to a list with 2‘—! elements, two more when the search has been reduced to a list with 
2k-2 elements, and so on, until two comparisons are used when the search has been reduced to a 
list with 2! = 2 elements. Finally, when one term is left in the list, one comparison tells us that 
there are no additional terms left, and one more comparison is used to determine if this term is x. 

Hence, at most 2k + 2 = 2logn + 2 comparisons are required to perform a binary search 
when the list being searched has 2‘ elements. (If n is not a power of 2, the original list is expanded 
to a list with 2+! terms, where k = [logn], and the search requires at most 2 [logn] +2 
comparisons.) It follows that in the worst case, binary search requires O(log) comparisons. 
Note that in the worst case, 2 logn + 2 comparisons are used by the binary search. Hence, the 
binary search uses ©(logn) comparisons in the worst case, because 2 logn + 2 = O(logn). 
From this analysis it follows that in the worst case, the binary search algorithm is more efficient 
than the linear search algorithm, because we know by Example 2 that the linear search algorithm 
has ©(n) worst-case time complexity. < 


AVERAGE-CASE COMPLEXITY Another important type of complexity analysis, besides 
worst-case analysis, is called average-case analysis. The average number of operations used to 
solve the problem over all possible inputs of a given size is found in this type of analysis. Average- 
case time complexity analysis is usually much more complicated than worst-case analysis. 


EXAMPLE 4 


EXAMPLE 5 
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However, the average-case analysis for the linear search algorithm can be done without difficulty, 
as shown in Example 4. 


Describe the average-case performance of the linear search algorithm in terms of the average 
number of comparisons used, assuming that the integer x is in the list and it is equally likely 
that x is in any position. 


Solution: By hypothesis, the integer x is one of the integers a1, a2, ..., Gp in the list. If x is the 
first term a, of the list, three comparisons are needed, one i < n to determine whether the end 
of the list has been reached, one x 4 a; to compare x and the first term, and one i < n outside 
the loop. If x is the second term az of the list, two more comparisons are needed, so that a total 
of five comparisons are used. In general, if x is the ith term of the list a;, two comparisons will 
be used at each of the i steps of the loop, and one outside the loop, so that a total of 2i + 1 
comparisons are needed. Hence, the average number of comparisons used equals 


3454274 OAH). COLTS Ei ayn 
n n ; 


Using the formula from line 2 of Table 2 in Section 2.4 (and see Exercise 37(b) of Section 2.4), 


n(n + 1) 


Le aot 


Hence, the average number of comparisons used by the linear search algorithm (when x is 
known to be in the list) is 


2[n(n + 1)/2] ae 
n 


which is (7). < 


Remark: In the analysis in Example 4 we assumed that x is in the list being searched. It is also 
possible to do an average-case analysis of this algorithm when x may not be in the list (see 
Exercise 23). 


Remark: Although we have counted the comparisons needed to determine whether we have 
reached the end of a loop, these comparisons are often not counted. From this point on we will 
ignore such comparisons. 


WORST-CASE COMPLEXITY OF TWO SORTING ALGORITHMS We analyze the 
worst-case complexity of the bubble sort and the insertion sort in Examples 5 and 6. 


What is the worst-case complexity of the bubble sort in terms of the number of comparisons 
made? 


Solution: The bubble sort described before Example 4 in Section 3.1 sorts a list by performing 
a sequence of passes through the list. During each pass the bubble sort successively compares 
adjacent elements, interchanging them if necessary. When the ith pass begins, the i — 1 largest 
elements are guaranteed to be in the correct positions. During this pass, n — i comparisons are 
used. Consequently, the total number of comparisons used by the bubble sort to order a list of 
n elements is 


Pb hee 
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EXAMPLE 6 


using a summation formula from line 2 in Table 2 in Section 2.4 (and Exercise 37(b) in 
Section 2.4). Note that the bubble sort always uses this many comparisons, because it con- 
tinues even if the list becomes completely sorted at some intermediate step. Consequently, the 
bubble sort uses (n — 1)n/2 comparisons, so it has @(n”) worst-case complexity in terms of 
the number of comparisons used. < 


What is the worst-case complexity of the insertion sort in terms of the number of comparisons 
made? 


Solution: The insertion sort (described in Section 3.1) inserts the jth element into the correct 
position among the first 7 — 1 elements that have already been put into the correct order. It does 
this by using a linear search technique, successively comparing the jth element with successive 
terms until a term that is greater than or equal to it is found or it compares a; with itself and stops 
because a; is not less than itself. Consequently, in the worst case, j comparisons are required 
to insert the jth element into the correct position. Therefore, the total number of comparisons 
used by the insertion sort to sort a list of n elements is 


n(n + 1) _ 


yee eee = 
a ae 5 


IF 

using the summation formula for the sum of consecutive integers in line 2 of Table 2 of 
Section 2.4 (and see Exercise 37(b) of Section 2.4), and noting that the first term, 1, is missing 
in this sum. Note that the insertion sort may use considerably fewer comparisons if the smaller 
elements started out at the end of the list. We conclude that the insertion sort has worst-case 
complexity @(n7). < 


In Examples 5 and 6 we showed that both the bubble sort and the insertion sort have 
worst-case time complexity ©(n”). However, the most efficient sorting algorithms can sort n 
items in O(n log n) time, as we will show in Sections 8.3 and 11.1 using techniques we develop in 
those sections. From this point on, we will assume that sorting 1 items can be done in O(n log n) 
time. 


Complexity of Matrix Multiplication 


The definition of the product of two matrices can be expressed as an algorithm for computing 
the product of two matrices. Suppose that C = [c;;] is the m x n matrix that is the product of the 
m x k matrix A = [a;;] and the k x n matrix B = [5;;]. The algorithm based on the definition 
of the matrix product is expressed in pseudocode in Algorithm 1. 


ALGORITHM 1 Matrix Multiplication. 


procedure matrix multiplication(A, B: matrices) 
fori := 1tom 
for j :=1ton 
Cij :=0 
for g := 1 tok 
Cij = Cij + Gighgj 
return C {C = [c;;] is the product of A and B} 


We can determine the complexity of this algorithm in terms of the number of additions and 
multiplications used. 


EXAMPLE 7 


EXAMPLE 8 


Links 


EXAMPLE 9 
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How many additions of integers and multiplications of integers are used by Algorithm | to 
multiply two n x n matrices with integer entries? 


Solution: There are n? entries in the product of A and B. To find each entry requires a total 
of n multiplications and n — 1 additions. Hence, a total of n> multiplications and n7(n — 1) 
additions are used. 


Surprisingly, there are more efficient algorithms for matrix multiplication than that given in 
Algorithm 1. As Example 7 shows, multiplying two n x n matrices directly from the definition 
requires O(n*) multiplications and additions. Using other algorithms, two n x n matrices can 
be multiplied using O(n?) multiplications and additions. (Details of such algorithms can be 
found in [CoLeRiSt09].) 

We can also analyze the complexity of the algorithm we described in Chapter 2 for computing 
the Boolean product of two matrices, which we display as Algorithm 2. 


ALGORITHM 2 The Boolean Product of Zero-One Matrices. 


procedure Boolean product of Zero-One Matrices (A, B: zero—one matrices) 
fori := 1tom 
for j := 1 ton 
Cij :=0 
for gq := 1 tok 
Cij = Cij V (dig ax bqj) 
return C {C = [c;;] is the Boolean product of A and B} 


The number of bit operations used to find the Boolean product of two n x n matrices can 
be easily determined. 


How many bit operations are used to find A © B, where A and B aren x n zero—one matrices? 


Solution: There are n? entries in A© B. Using Algorithm 2, a total of n ORs and n ANDs are 
used to find an entry of A © B. Hence, 2n bit operations are used to find each entry. Therefore, 
2n? bit operations are required to compute A © B using Algorithm 2. 


MATRIX-CHAIN MULTIPLICATION There is another important problem involving the 
complexity of the multiplication of matrices. How should the matrix-chain A; A) - - - A, be com- 
puted using the fewest multiplications of integers, where Aj, Ao, ..., An are my; X m2,m2 X 
M3,..., Mn X My+1 Matrices, respectively, and each has integers as entries? (Because matrix 
multiplication is associative, as shown in Exercise 13 in Section 2.6, the order of the mul- 
tiplication used does not change the product.) Note that m,m2m3 multiplications of integers 
are performed to multiply an m, x m2 matrix and an m2 x m3 matrix using Algorithm 1. 
Example 9 illustrates this problem. 


In which order should the matrices Aj, Az, and Az3—where A, is 30 x 20, Ag is 20 x 40, and 
A; is 40 x 10, all with integer entries—be multiplied to use the least number of multiplications 
of integers? 


Solution: There are two possible ways to compute A; A2A3. These are A; (A2A3) and (A; A2)A3. 

If Az and Az; are first multiplied, a total of 20 - 40 - 10 = 8000 multiplications of inte- 
gers are used to obtain the 20 x 10 matrix A2A3. Then, to multiply A; and A2A3 requires 
30 - 20 - 10 = 6000 multiplications. Hence, a total of 


8000 + 6000 = 14,000 
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multiplications are used. On the other hand, if A; and Az are first multiplied, then 30 - 20 - 40 = 
24,000 multiplications are used to obtain the 30 x 40 matrix A;A2. Then, to multiply A;A2 
and A3 requires 30 - 40 - 10 = 12,000 multiplications. Hence, a total of 


24,000 + 12,000 = 36,000 


multiplications are used. 
Clearly, the first method is more efficient. < 


We will return to this problem in Exercise 57 in Section 8.1. Algorithms for determining 
the most efficient way to carry out matrix-chain multiplication are discussed in [CoLeRiSt09]. 


Algorithmic Paradigms 


In Section 3.1 we introduced the basic notion of an algorithm. We provided examples of many 
different algorithms, including searching and sorting algorithms. We also introduced the concept 
of a greedy algorithm, giving examples of several problems that can be solved by greedy algo- 
rithms. Greedy algorithms provide an example of an algorithmic paradigm, that is, a general 
approach based on a particular concept that can be used to construct algorithms for solving a 
variety of problems. 

In this book we will construct algorithms for solving many different problems based on a 
variety of algorithmic paradigms, including the most widely used algorithmic paradigms. These 
paradigms can serve as the basis for constructing efficient algorithms for solving a wide range 
of problems. 

Some of the algorithms we have already studied are based on an algorithmic paradigm known 
as brute force, which we will describe in this section. Algorithmic paradigms, studied later in 
this book, include divide-and-conquer algorithms studied in Chapter 8, dynamic programming, 
also studied in Chapter 8, backtracking, studied in Chapter 10, and probabilistic algorithms, 
studied in Chapter 7. There are many important algorithmic paradigms besides those described 
in this book. Consult books on algorithm design such as [KITa06] to learn more about them. 


BRUTE-FORCE ALGORITHMS Brute force is an important, and basic, algorithmic 
paradigm. In a brute-force algorithm, a problem is solved in the most straightforward manner 
based on the statement of the problem and the definitions of terms. Brute-force algorithms are 
designed to solve problems without regard to the computing resources required. For example, 
in some brute-force algorithms the solution to a problem is found by examining every possible 
solution, looking for the best possible. In general, brute-force algorithms are naive approaches 
for solving problems that do not take advantage of any special structure of the problem or clever 
ideas. 

Note that Algorithm | in Section 3.1 for finding the maximum number in a sequence is 
a brute-force algorithm because it examines each of the n numbers in a sequence to find the 
maximum term. The algorithm for finding the sum of n numbers by adding one additional 
number at a time is also a brute-force algorithm, as is the algorithm for matrix multiplication 
based on its definition (Algorithm 1). The bubble, insertion, and selection sorts (described in 
Section 3.1 in Algorithms 4 and 5 and in Exercise 42, respectively) are also considered to be 
brute-force algorithms; all three of these sorting algorithms are straightforward approaches much 
less efficient than other sorting algorithms such as the merge sort and the quick sort discussed 
in Chapters 5 and 8. 

Although brute-force algorithms are often inefficient, they are often quite useful. A brute- 
force algorithm may be able to solve practical instances of problems, particularly when the input 


EXAMPLE 10 
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is not too large, even if it is impractical to use this algorithm for larger inputs. Furthermore, 
when designing new algorithms to solve a problem, the goal is often to find a new algorithm 
that is more efficient than a brute-force algorithm. One such problem of this type is described 
in Example 10. 


Construct a brute-force algorithm for finding the closest pair of points in a set of n points in 
the plane and provide a worst-case big-O estimate for the number of bit operations used by the 
algorithm. 


Solution: Suppose that we are given as input the points (x1, y,), (X2, y2),---, (Xn, Yn). Recall 


that the distance between (x;, yj) and (xj, y;) is (i — x)? + (yj - y;)2. A brute-force algo- 
rithm can find the closest pair of these points by computing the distances between all pairs of 
the n points and determining the smallest distance. (We can make one small simplification to 
make the computation easier; we can compute the square of the distance between pairs of points 
to find the closest pair, rather than the distance between these points. We can do this because 
the square of the distance between a pair of points is smallest when the distance between these 
points is smallest.) 


ALGORITHM 3 Brute-Force Algorithm for Closest Pair of Points. 


procedure closest-pair((x,, y1), (x2, y2),---, (Xn, Yn): pairs of real numbers) 
min= oo 
for i :=2ton 
for j := 1toi-1 
if (x; — xi)? + Oj - yi)? < min then 

min := (xj — xj)? + Qj — yi)? 

closest pair := ((x;, yi), (Xj, Yj) 
return closest pair 


To estimate the number of operations used by the algorithm, first note that there are 
n(n — 1)/2 pairs of points ((x;, y;), (xj, y;)) that we loop through (as the reader should verify). 
For each such pair we compute (x; — xi)? + (yj - y;)*, compare it with the current value of 
min, and if it is smaller than min replace the current value of min by this new value. It follows 
that this algorithm uses ©(n7) operations, in terms of arithmetic operations and comparisons. 

In Chapter 8 we will devise an algorithm that determines the closest pair of points when given 
n points in the plane as input that has O(n log) worst-case complexity. The original discovery 
of such an algorithm, much more efficient than the brute-force approach, was considered quite 
surprising. < 


Understanding the Complexity of Algorithms 


Table | displays some common terminology used to describe the time complexity of algorithms. 
For example, an algorithm that finds the largest of the first 100 terms of a list of n elements 
by applying Algorithm 1 to the sequence of the first 100 terms, where n is an integer with 
n > 100, has constant complexity because it uses 99 comparisons no matter what n is (as 
the reader can verify). The linear search algorithm has linear (worst-case or average-case) 
complexity and the binary search algorithm has logarithmic (worst-case) complexity. Many 
important algorithms have n log n, or linearithmic (worst-case) complexity, such as the merge 
sort, which we will introduce in Chapter 4. (The word linearithmic is a combination of the words 
linear and logarithmic.) 
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TABLE 1 Commonly Used Terminology for the 
Complexity of Algorithms. 


Complexity Terminology 


0d) Constant complexity 
O(log n) Logarithmic complexity 
O(n) Linear complexity 

O(n log n) Linearithmic complexity 


@(n’) Polynomial complexity 


@(b"), where b > 1 Exponential complexity 
O(n!) Factorial complexity 


An algorithm has polynomial complexity if it has complexity @(n”), where b is an integer 
with b > 1. For example, the bubble sort algorithm is a polynomial-time algorithm because 
it uses ©(n7) comparisons in the worst case. An algorithm has exponential complexity if it 
has time complexity ©(b"), where b > 1. The algorithm that determines whether a compound 
proposition in n variables is satisfiable by checking all possible assignments of truth variables 
is an algorithm with exponential complexity, because it uses ©(2”) operations. Finally, an 
algorithm has factorial complexity if it has © (n!) time complexity. The algorithm that finds all 
orders that a traveling salesperson could use to visit n cities has factorial complexity; we will 
discuss this algorithm in Chapter 9. 


TRACTABILITY A problem that is solvable using an algorithm with polynomial worst-case 
complexity is called tractable, because the expectation is that the algorithm will produce the 
solution to the problem for reasonably sized input in a relatively short time. However, if the 
polynomial in the big-© estimate has high degree (such as degree 100) or if the coefficients 
are extremely large, the algorithm may take an extremely long time to solve the problem. 
Consequently, that a problem can be solved using an algorithm with polynomial worst-case 
time complexity is no guarantee that the problem can be solved in a reasonable amount of time 
for even relatively small input values. Fortunately, in practice, the degree and coefficients of 
polynomials in such estimates are often small. 

The situation is much worse for problems that cannot be solved using an algorithm with 
worst-case polynomial time complexity. Such problems are called intractable. Usually, but not 
always, an extremely large amount of time is required to solve the problem for the worst cases 
of even small input values. In practice, however, there are situations where an algorithm with a 
certain worst-case time complexity may be able to solve a problem much more quickly for most 
cases than for its worst case. When we are willing to allow that some, perhaps small, number 
of cases may not be solved in a reasonable amount of time, the average-case time complexity is 
a better measure of how long an algorithm takes to solve a problem. Many problems important 
in industry are thought to be intractable but can be practically solved for essentially all sets of 
input that arise in daily life. Another way that intractable problems are handled when they arise 
in practical applications is that instead of looking for exact solutions of a problem, approximate 
solutions are sought. It may be the case that fast algorithms exist for finding such approximate so- 
lutions, perhaps even with a guarantee that they do not differ by very much from an exact solution. 

Some problems even exist for which it can be shown that no algorithm exists for solving 
them. Such problems are called unsolvable (as opposed to solvable problems that can be 
solved using an algorithm). The first proof that there are unsolvable problems was provided by 
the great English mathematician and computer scientist Alan Turing when he showed that the 
halting problem is unsolvable. Recall that we proved that the halting problem is unsolvable in 
Section 3.1. (A biography of Alan Turing and a description of some of his other work can be 
found in Chapter 13.) 
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P VERSUS NP The study of the complexity of algorithms goes far beyond what we can 
describe here. Note, however, that many solvable problems are believed to have the property 
that no algorithm with polynomial worst-case time complexity solves them, but that a solution, 
if known, can be checked in polynomial time. Problems for which a solution can be checked 
in polynomial time are said to belong to the class NP (tractable problems are said to belong to 
class P). The abbreviation NP stands for nondeterministic polynomial time. The satisfiability 
problem, discussed in Section 1.3, is an example of an NP problem—we can quickly verify that 
an assignment of truth values to the variables of a compound proposition makes it true, but no 
polynomial time algorithm has been discovered for finding such an assignment of truth values. 
(For example, an exhaustive search of all possible truth values requires (Q2(2”) bit operations 
where 7 is the number of variables in the compound proposition.) 

There is also an important class of problems, called NP-complete problems, with the 

property that if any of these problems can be solved by a polynomial worst-case time algorithm, 
links Bd then all problems in the class NP can be solved by polynomial worst-case time algorithms. 
The satisfiability problem, is also an example of an NP-complete problem. It is an NP problem 
and if a polynomial time algorithm for solving it were known, there would be polynomial time 
algorithms for all problems known to be in this class of problems (and there are many important 
problems in this class). This last statement follows from the fact that every problem in NP 
can be reduced in polynomial time to the satisfiability problem. Although more than 3000 NP- 
complete problems are now known, the satisfiability problem was the first problem shown to be 
NP-complete. The theorem that asserts this is known as the Cook-Levin theorem after Stephen 

Cook and Leonid Levin, who independently proved it in the early 1970s. 

The P versus NP problem asks whether NP, the class of problems for which it is possible 
to check solutions in polynomial time, equals P, the class of tractable problems. If PANP, there 
would be some problems that cannot be solved in polynomial time, but whose solutions could 
be verified in polynomial time. The concept of NP-completeness is helpful in research aimed 

® at solving the P versus NP problem, because NP-complete problems are the problems in NP 
considered most likely not to be in P, as every problem in NP can be reduced to an NP-complete 
problem in polynomial time. A large majority of theoretical computer scientists believe that 
P 4 NP, which would mean that no NP-complete problem can be solved in polynomial time. 
One reason for this belief is that despite extensive research, no one has succeeded in showing that 
: P=NP. In particular, no one has been able to find an algorithm with worst-case polynomial time 
Links Qi complexity that solves any NP-complete problem. The P versus NP problem is one of the most 
famous unsolved problems in the mathematical sciences (which include theoretical computer 
science). It is one of the seven famous Millennium Prize Problems, of which six remain unsolved. 

A prize of $1,000,000 is offered by the Clay Mathematics Institute for its solution. 


STEPHEN COOK (BORN 1939) Stephen Cook was born in Buffalo where his father worked as an industrial 
chemist and taught university courses. His mother taught English courses in a community college. While in 
high school Cook developed an interest in electronics through his work with a famous local inventor noted for 
inventing the first implantable cardiac pacemaker. 

Cook was a mathematics major at the University of Michigan, graduating in 1961. He did graduate work 
at Harvard, receiving a master’s degree in 1962 and a Ph.D. in 1966. Cook was appointed an assistant professor 
in the Mathematics Department at the University of California, Berkeley in 1966. He was not granted tenure 
there, possibly because the members of the Mathematics Department did not find his work on what is now 
considered to be one of the most important areas of theoretical computer science of sufficient interest. In 1970, 
he joined the University of Toronto as an assistant professor, holding a joint appointment in the Computer 
Science Department and the Mathematics Department. He has remained at the University of Toronto, where he was appointed a 
University Professor in 1985. 

Cook is considered to be one of the founders of computational complexity theory. His 1971 paper “The Complexity of Theorem 
Proving Procedures” formalized the notions of NP-completeness and polynomial-time reduction, showed that NP-complete problems 
exist by showing that the satisfiability problem is such a problem, and introduced the notorious P versus NP problem. 

Cook has received many awards, including the 1982 Turing Award. He is married and has two sons. Among his interests are 
playing the violin and racing sailboats. 
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For more information about the complexity of algorithms, consult the references, including 
[CoLeRiSt09], for this section listed at the end of this book. (Also, for a more formal discussion 
of computational complexity in terms of Turing machines, see Section 13.5.) 


PRACTICAL CONSIDERATIONS _ Note that a big-© estimate of the time complexity of an 
algorithm expresses how the time required to solve the problem increases as the input grows 
in size. In practice, the best estimate (that is, with the smallest reference function) that can be 
shown is used. However, big-© estimates of time complexity cannot be directly translated into 
the actual amount of computer time used. One reason is that a big-© estimate f(n) is O(g(n)), 
where f(n) is the time complexity of an algorithm and g(n) is a reference function, means 
that C)g(n) < f(n) < Cog(n) when n > k, where Cj, C2, and k are constants. So without 
knowing the constants C;, C2, and k in the inequality, this estimate cannot be used to determine 
a lower bound and an upper bound on the number of operations used in the worst case. As 
remarked before, the time required for an operation depends on the type of operation and the 
computer being used. Often, instead of a big-© estimate on the worst-case time complexity of 
an algorithm, we have only a big-O estimate. Note that a big-O estimate on the time complexity 
of an algorithm provides an upper, but not a lower, bound on the worst-case time required for 
the algorithm as a function of the input size. Nevertheless, for simplicity, we will often use 
big-O estimates when describing the time complexity of algorithms, with the understanding 
that big-© estimates would provide more information. 

Table 2 displays the time needed to solve problems of various sizes with an algorithm using 
the indicated number n of bit operations, assuming that each bit operation takes 10~!! seconds, a 
reasonable estimate of the time required for a bit operation using the fastest computers available 
today. Times of more than 10! years are indicated with an asterisk. In the future, these times 
will decrease as faster computers are developed. We can use the times shown in Table 2 to see 
whether it is reasonable to expect a solution to a problem of a specified size using an algorithm 
with known worst-case time complexity when we run this algorithm on a modern computer. 
Note that we cannot determine the exact time a computer uses to solve a problem with input of 
a particular size because of a myriad of issues involving computer hardware and the particular 
software implementation of the algorithm. 

It is important to have a reasonable estimate for how long it will take a computer to solve a 
problem. For instance, if an algorithm requires approximately 10 hours, it may be worthwhile to 
spend the computer time (and money) required to solve this problem. But, if an algorithm requires 
approximately 10 billion years to solve a problem, it would be unreasonable to use resources to 
implement this algorithm. One of the most interesting phenomena of modern technology is the 
tremendous increase in the speed and memory space of computers. Another important factor 
that decreases the time needed to solve problems on computers is parallel processing, which 
is the technique of performing sequences of operations simultaneously. 

Efficient algorithms, including most algorithms with polynomial time complexity, benefit 
most from significant technology improvements. However, these technology improvements 


TABLE 2 The Computer Time Used by Algorithms. 
Bit Operations Used 


10-9 s 


1077 5 
107 s 
1073 s 
0.1 s 
0.17 min 
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offer little help in overcoming the complexity of algorithms of exponential or factorial time 
complexity. Because of the increased speed of computation, increases in computer memory, and 
the use of algorithms that take advantage of parallel processing, many problems that were con- 
sidered impossible to solve five years ago are now routinely solved, and certainly five years from 
now this statement will still be true. This is even true when the algorithms used are intractable. 


Exercises 


. Give a big-O estimate for the number of operations 
(where an operation is an addition or a multiplication) 
used in this segment of an algorithm. 


t:=0 

for i := 1 to3 
for j := 1to4 
ti=t+ij 


. Give a big-O estimate for the number additions used in 
this segment of an algorithm. 


t:=0 

fori :=1ton 
for j :=1ton 
ti=t+it+] 


. Give a big-O estimate for the number of operations, 
where an operation is a comparison or a multiplication, 
used in this segment of an algorithm (ignoring compar- 
isons used to test the conditions in the for loops, where 
a1, 42, ..-, An are positive real numbers). 

m:=0 

for i := 1 ton 

for j:=i+l1ton 
m := max(ajaj;,m) 


. Give a big-O estimate for the number of operations, 
where an operation is an addition or a multiplication, used 
in this segment of an algorithm (ignoring comparisons 
used to test the conditions in the while loop). 


i:=1 

t:=0 

while i <n 
t:=tt+i 
i:= 2i 


. How many comparisons are used by the algorithm given 
in Exercise 16 of Section 3.1 to find the smallest natural 
number in a sequence of n natural numbers? 

. a) Use pseudocode to describe the algorithm that puts the 
first four terms of a list of real numbers of arbitrary 
length in increasing order using the insertion sort. 

b) Show that this algorithm has time complexity O(1) in 
terms of the number of comparisons used. 

. Suppose that an element is known to be among the first 

four elements in a list of 32 elements. Would a lin- 

ear search or a binary search locate this element more 
rapidly? 

. Given a real number x and a positive integer k, determine 

the number of multiplications used to find x2" starting 


*10. 


11. 


12. 


with x and successively squaring (to find =, x‘, and so 
on). Is this a more efficient way to find x2. than by mul- 
tiplying x by itself the appropriate number of times? 


. Give a big-O estimate for the number of comparisons 


used by the algorithm that determines the number of 1s 
in a bit string by examining each bit of the string to deter- 
mine whether it is a 1 bit (see Exercise 25 of Section 3.1). 


a) Show that this algorithm determines the number of 1 
bits in the bit string S: 


procedure bit count(S: bit string) 


count :=0 

while S 4 0 
count := count + 1 
S:=SA(S—1) 


return count {count is the number of 1s in S} 


Here S — 1 is the bit string obtained by changing the 
rightmost | bit of S to a 0 and all the 0 bits to the right 
of this to 1s. [Recall that § A (S — 1) is the bitwise 
AND of S and S — 1.] 

How many bitwise AND operations are needed to find 
the number of | bits in a string S using the algorithm 
in part (a)? 


b 


eS 


a) Suppose we have n subsets $1, S2,..., S, of the set 
{1,2,...,}. Express a brute-force algorithm that de- 
termines whether there is a disjoint pair of these sub- 
sets. [Hint: The algorithm should loop through the 
subsets; for each subset S;, it should then loop through 
all other subsets; and for each of these other subsets 
Sj, it should loop through all elements k in S; to de- 
termine whether k also belongs to S;.] 

Give a big-O estimate for the number of times the 
algorithm needs to determine whether an integer is in 
one of the subsets. 


b 


SS 


Consider the following algorithm, which takes as input a 
sequence of n integers a1, a2, ..., A, and produces as out- 
put a matrix M = {m;;} where m;; is the minimum term 
in the sequence of integers a;, aj41,..., aj for j > i and 
mj; = 0 otherwise. 
initialize M so that mj; = a; if j = i and m;; = 0 
otherwise 
fori := 1 ton 
for j:=i+1ton 
fork :=i+1toj 
mij = min(mN;;, ag) 
return M= {m;;} {mj; is the minimum term of 
qj, Gj+1, 1 aj} 
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13. 


14. 


15. 


16. 


a) Show that this algorithm uses O(n*) comparisons to 
compute the matrix M. 

b) Show that this algorithm uses Q(n*) comparisons to 
compute the matrix M. Using this fact and part (a), 
conclude that the algorithms uses © (n?) comparisons. 
[Hint: Only consider the cases where i < n/4 and 
j = 3n/4 in the two outer loops in the algorithm. ] 

The conventional algorithm for evaluating a polynomial 

Ayx" + dy—x"—! +--+ + a,x +a at x =c can be ex- 

pressed in pseudocode by 


procedure polynomial(c, ap, a1, ...,@,: real numbers) 
power := 1 
y:= a0 


for i := 1 ton 
power := power * C 
y i= y +4; * power 
return y {y = a,c" + dn—1c"—! +++» + ajc +.ao} 


where the final value of y is the value of the polynomial 
atx =c. 


a) Evaluate 3x2 + x +1 at x =2 by working through 
each step of the algorithm showing the values assigned 
at each assignment step. 

b) Exactly how many multiplications and additions are 
used to evaluate a polynomial of degree n at x = c? 
(Do not count additions used to increment the loop 
variable.) 


There is a more efficient algorithm (in terms of the num- 
ber of multiplications and additions used) for evaluating 
polynomials than the conventional algorithm described 
in the previous exercise. It is called Horner’s method. 
This pseudocode shows how to use this method to find the 


value of a,x" + dy_jx"~! +---+ayx tag atx =c. 


procedure Horner(c, ao, a1, 42, ... 
y= an 
fori := 1 ton 
yi=y*C+ ani 
return y {y = ajc" +a,_\c"~! +--+» +.ayc +a} 


, Ay: real numbers) 


a) Evaluate 3x2 +x +1 at x =2 by working through 
each step of the algorithm showing the values assigned 
at each assignment step. 

b) Exactly how many multiplications and additions are 
used by this algorithm to evaluate a polynomial of 
degree n at x = c? (Do not count additions used to 
increment the loop variable.) 

What is the largest n for which one can solve within one 

second a problem using an algorithm that requires f (n) 

bit operations, where each bit operation is carried out in 

1079 seconds, with these functions f (n)? 


a) logn b) n c) nlogn 
d) n?2 e) 2” f) n! 
What is the largest n for which one can solve within a 


day using an algorithm that requires f (n) bit operations, 
where each bit operation is carried out in 107!! seconds, 
with these functions f (n)? 


17. 


18. 


19, 


20. 


21. 


22. 


23. 


24. 


a) logn b) 1000n ¢) nr? 

d) 1000n? e) n° £).2" 

g) 22" h) 22" 

What is the largest n for which one can solve within 


a minute using an algorithm that requires f(n) bit op- 
erations, where each bit operation is carried out in 
1012 seconds, with these functions f(n)? 


a) loglogn b) logn c) (log ny? 

d) 1000000n e) n° f) 2” 

g) 2” 

How much time does an algorithm take to solve a prob- 


lem of size n if this algorithm uses 2n? + 2” operations, 
each requiring 10~° seconds, with these values of n? 
a) 10 b) 20 c) 50 d) 100 


How much time does an algorithm using 2°° operations 
need if each operation takes these amounts of time? 

a) 102° s b) 10-°s e) 10-5 
What is the effect in the time required to solve a prob- 
lem when you double the size of the input from n to 2n, 
assuming that the number of milliseconds the algorithm 
uses to solve the problem with input size n is each of these 
function? [Express your answer in the simplest form pos- 
sible, either as a ratio or a difference. Your answer may 
be a function of n or a constant.] 


a) loglogn b) logn c) 100n 

d) nlogn e) nr? f) n? 

g) 2" 

What is the effect in the time required to solve a problem 


when you increase the size of the input from n ton + 1, 
assuming that the number of milliseconds the algorithm 
uses to solve the problem with input size n is each of these 
function? [Express your answer in the simplest form pos- 
sible, either as a ratio or a difference. Your answer may 
be a function of n or a constant.] 


a) logn b) 100n c) nn? 

d) 3 e) 2” f) 2” 

g) n! 

Determine the least number of comparisons, or best-case 


performance, 


a) required to find the maximum of a sequence of n in- 
tegers, using Algorithm 1 of Section 3.1. 


b) used to locate an element in a list of n terms with a 
linear search. 


c) used to locate an element in a list of n terms using a 
binary search. 

Analyze the average-case performance of the linear 

search algorithm, if exactly half the time the element x is 

not in the list and if x is in the list it is equally likely to 

be in any position. 


An algorithm is called optimal for the solution of a prob- 
lem with respect to a specified operation if there is no 
algorithm for solving this problem using fewer opera- 
tions. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


a) Show that Algorithm 1 in Section 3.1 is an optimal 
algorithm with respect to the number of comparisons 
of integers. [Vote: Comparisons used for bookkeep- 
ing in the loop are not of concern here.] 

b) Is the linear search algorithm optimal with respect to 
the number of comparisons of integers (not including 
comparisons used for bookkeeping in the loop)? 


Describe the worst-case time complexity, measured in 
terms of comparisons, of the ternary search algorithm 
described in Exercise 27 of Section 3.1. 

Describe the worst-case time complexity, measured in 
terms of comparisons, of the search algorithm described 
in Exercise 28 of Section 3.1. 

Analyze the worst-case time complexity of the algorithm 
you devised in Exercise 29 of Section 3.1 for locating a 
mode in a list of nondecreasing integers. 

Analyze the worst-case time complexity of the algorithm 
you devised in Exercise 30 of Section 3.1 for locating all 
modes in a list of nondecreasing integers. 

Analyze the worst-case time complexity of the algorithm 
you devised in Exercise 31 of Section 3.1 for finding the 
first term of a sequence of integers equal to some previous 
term. 

Analyze the worst-case time complexity of the algorithm 
you devised in Exercise 32 of Section 3.1 for finding all 
terms of a sequence that are greater than the sum of all 
previous terms. 

Analyze the worst-case time complexity of the algorithm 
you devised in Exercise 33 of Section 3.1 for finding the 
first term of a sequence less than the immediately preced- 
ing term. 

Determine the worst-case complexity in terms of com- 
parisons of the algorithm from Exercise 5 in Section 3.1 
for determining all values that occur more than once in a 
sorted list of integers. 

Determine the worst-case complexity in terms of compar- 
isons of the algorithm from Exercise 9 in Section 3.1 for 
determining whether a string of n characters is a palin- 
drome. 

How many comparisons does the selection sort (see 
preamble to Exercise 41 in Section 3.1) use to sort n 
items? Use your answer to give a big-O estimate of the 
complexity of the selection sort in terms of number of 
comparisons for the selection sort. 

Find a big-O estimate for the worst-case complexity in 
terms of number of comparisons used and the number of 
terms swapped by the binary insertion sort described in 
the preamble to Exercise 47 in Section 3.1. 

Show that the greedy algorithm for making change for n 
cents using quarters, dimes, nickels, and pennies has O(n) 
complexity measured in terms of comparisons needed. 


Exercises 37 and 38 deal with the problem of scheduling the 
most talks possible given the start and end times of 7 talks. 


37. 


Find the complexity of a brute-force algorithm for 
scheduling the talks by examining all possible subsets 
of the talks. [Hint: Use the fact that a set with n elements 
has 2” subsets. ] 


38. 


39. 


40. 
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Find the complexity of the greedy algorithm for schedul- 
ing the most talks by adding at each step the talk with the 
earliest end time compatible with those already scheduled 
(Algorithm 7 in Section 3.1). Assume that the talks are 
not already sorted by earliest end time and assume that 
the worst-case time complexity of sorting is O(n logn). 


Describe how the number of comparisons used in the 
worst case changes when these algorithms are used to 
search for an element of a list when the size of the list 
doubles from n to 2n, where n is a positive integer. 


a) linear search b) binary search 


Describe how the number of comparisons used in the 
worst case changes when the size of the list to be sorted 
doubles from n to 2n, where n is a positive integer when 
these sorting algorithms are used. 

a) bubble sort b) insertion sort 


c) selection sort (described in the preamble to Exer- 
cise 41 in Section 3.1) 

d) binary insertion sort (described in the preamble to Ex- 
ercise 47 in Section 3.1) 


Ann X n matrix is called upper triangular if a;; = 0 when- 
everi > j. 


41. 


42. 


43. 


From the definition of the matrix product, describe an 
algorithm in English for computing the product of two 
upper triangular matrices that ignores those products in 
the computation that are automatically equal to zero. 


Give a pseudocode description of the algorithm in Exer- 
cise 41 for multiplying two upper triangular matrices. 


How many multiplications of entries are used by the al- 
gorithm found in Exercise 41 for multiplying two n x n 
upper triangular matrices? 


In Exercises 44—45 assume that the number of multiplications 
of entries used to multiply a p x g matrix andaq x r matrix 


is pqr. 


44. 


45. 


* 46. 


What is the best order to form the product ABC if A, B, 
and C are matrices with dimensions 3 x 9, 9 x 4, and 
4 x 2, respectively? 


What is the best order to form the product ABCD if A, B, 
C, and D are matrices with dimensions 30 x 10,10 x 40, 
40 x 50, and 50 x 30, respectively?. 


In this exercise we deal with the problem of string match- 

ing. 

a) Explain how to use a brute-force algorithm to find 
the first occurrence of a given string of m characters, 
called the target, in a string of n characters, where 
m <n, called the text. [Hint: Think in terms of find- 
ing a match for the first character of the target and 
checking successive characters for a match, and if 
they do not all match, moving the start location one 
character to the right.] 


b) Express your algorithm in pseudocode. 


c) Give a big-O estimate for the worst-case time com- 
plexity of the brute-force algorithm you described. 
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Key Terms and Results 


TERMS 


algorithm: a finite sequence of precise instructions for per- 
forming a computation or solving a problem 

searching algorithm: the problem of locating an element in a 
list 

linear search algorithm: a procedure for searching a list ele- 
ment by element 

binary search algorithm: a procedure for searching an or- 
dered list by successively splitting the list in half 

sorting: the reordering of the elements of a list into prescribed 
order 

f() is O(g(x)): the fact that | f(x)| < Clg(x)| for all x >k 
for some constants C and k 

witness to the relationship f(x) is O(g(x)): a pair C and k 
such that | f (x)| < C|g(x)| whenever x > k 

F(x) is Q(g(x)): the fact that | f(x)| > C|g(x)| for all x > k 
for some positive constants C and k 

f(x) is O(g(x)): the fact that f(x) is both O(g(x)) and Q(g(x)) 

time complexity: the amount of time required for an algorithm 
to solve a problem 

space complexity: the amount of space in computer memory 
required for an algorithm to solve a problem 

worst-case time complexity: the greatest amount of time re- 
quired for an algorithm to solve a problem of a given size 

average-case time complexity: the average amount of time 
required for an algorithm to solve a problem of a given size 

algorithmic paradigm: a general approach for constructing 
algorithms based on a particular concept 

brute force: the algorithmic paradigm based on constructing 
algorithms for solving problems in a naive manner from the 
statement of the problem and definitions 


Review Questions 


1. a) Define the term algorithm. 
b) What are the different ways to describe algorithms? 
c) What is the difference between an algorithm for solv- 
ing a problem and a computer program that solves this 
problem? 
2. a) Describe, using English, an algorithm for finding the 
largest integer in a list of n integers. 
b) Express this algorithm in pseudocode. 


c) How many comparisons does the algorithm use? 


3. a) State the definition of the fact that f(n) is O(g(n)), 
where f(n) and g(n) are functions from the set of 
positive integers to the set of real numbers. 

b) Use the definition of the fact that f(n) is O(g(n)) 
directly to prove or disprove that n* + 18n + 107 is 
O(n). 

c) Use the definition of the fact that f(n) is 
O(g(n)) directly to prove or disprove that n° is 
O(n? + 18n + 107). 


greedy algorithm: an algorithm that makes the best choice at 
each step according to some specified condition 

tractable problem: a problem for which there is a worst-case 
polynomial-time algorithm that solves it 

intractable problem: a problem for which no worst-case 
polynomial-time algorithm exists for solving it 

solvable problem: a problem that can be solved by an algo- 
rithm 

unsolvable problem: a problem that cannot be solved by an 
algorithm 


RESULTS 


linear and binary search algorithms: (given in Section 3.1) 

bubble sort: a sorting that uses passes where successive items 
are interchanged if they in the wrong order 

insertion sort: a sorting that at the jth step inserts the jth el- 
ement into the correct position in in the list, when the first 
j — 1 elements of the list are already sorted 


The linear search has O(n) worst case time complexity. 

The binary search has O(log n) worst case time complexity. 

The bubble and insertion sorts have O(n) worst case time 
complexity. 

logn! is O(n logn). 

If fi (x) is O(g1 (x)) and fo (x) is O(g2(x)), then (fi + f2)(x) 
is O(max(g1 (x), g2(x))) and (fi f2)() is O((g1g2(%)). 

If ao, a1, ..., G are real numbers with a, 4 0, then ay,x” + 
nx"! +--+ + a,x + ao is O(x”), and hence O(n) and 
Qn). 


4. List these functions so that each function is big-O of 
the next function in the list: Jogn)?, n3/1000000, ./n, 
100n + 101, 3”, n!, 2"n?. 


5. a) How can you produce a big-O estimate for a function 
that is the sum of different terms where each term is 
the product of several functions? 


b) Give a big-O estimate for the function f(n) = 
(al! + 1)(27 +1) + (n”-? + 8n"-3)(n3 + 2"). For 
the function g in your estimate f(x) is O(g(x)) use a 
simple function of smallest possible order. 


6. a) Define what the worst-case time complexity, average- 
case time complexity, and best-case time complexity 
(in terms of comparisons) mean for an algorithm that 
finds the smallest integer in a list of n integers. 


b) What are the worst-case, average-case, and best-case 
time complexities, in terms of comparisons, of the al- 
gorithm that finds the smallest integer in a list of n 
integers by comparing each of the integers with the 
smallest integer found so far? 


7. 


a) Describe the linear search and binary search algorithm 
for finding an integer in a list of integers in increasing 
order. 

b) Compare the worst-case time complexities of these 
two algorithms. 

c) Is one of these algorithms always faster than the other 
(measured in terms of comparisons)? 


. a) Describe the bubble sort algorithm. 


b) Use the bubble sort algorithm to sort the list 5, 2, 4, 
1, 3: 

c) Give a big-O estimate for the number of comparisons 
used by the bubble sort. 


9. a) Describe the insertion sort algorithm. 


Supplementary Exercises 


1. 


a) Describe an algorithm for locating the last occurrence 
of the largest number in a list of integers. 


b) Estimate the number of comparisons used. 


. a) Describe an algorithm for finding the first and second 


largest elements in a list of integers. 
b) Estimate the number of comparisons used. 


. a) Give an algorithm to determine whether a bit string 


contains a pair of consecutive zeros. 
b) How many comparisons does the algorithm use? 


. a) Suppose that a list contains integers that are in order 


of largest to smallest and an integer can appear repeat- 
edly in this list. Devise an algorithm that locates all 
occurrences of an integer x in the list. 


b) Estimate the number of comparisons used. 


. a) Adapt Algorithm 1 in Section 3.1 to find the maxi- 


mum and the minimum of a sequence of n elements 
by employing a temporary maximum and a temporary 
minimum that is updated as each successive element 
is examined. 


b) Describe the algorithm from part (a) in pseudocode. 

c) How many comparisons of elements in the sequence 
are carried out by this algorithm? (Do not count com- 
parisons used to determine whether the end of the se- 
quence has been reached.) 


. a) Describe in detail (and in English) the steps of an al- 


gorithm that finds the maximum and minimum of a 
sequence of n elements by examining pairs of suc- 
cessive elements, keeping track of a temporary maxi- 
mum and a temporary minimum. If n is odd, both the 
temporary maximum and temporary minimum should 
initially equal the first term, and if n is even, the tem- 
porary minimum and temporary maximum should be 
found by comparing the initial two elements. The tem- 
porary maximum and temporary minimum should be 
updated by comparing them with the maximum and 
minimum of the pair of elements being examined. 


b) Express the algorithm described in part (a) in pseu- 
docode. 


10. 


11. 


*7, 


10. 
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b) Use the insertion sort algorithm to sort the list 2, 5, 1, 
4,3. 

c) Give a big-O estimate for the number of comparisons 
used by the insertion sort. 


a) Explain the concept of a greedy algorithm. 

b) Provide an example of a greedy algorithm that pro- 
duces an optimal solution and explain why it produces 
an optimal solution. 

c) Provide an example of a greedy algorithm that does 
not always produce an optimal solution and explain 
why it fails to do so. 


Define what it means for a problem to be tractable and 
what it means for a problem to be solvable. 


c) How many comparisons of elements of the sequence 
are carried out by this algorithm? (Do not count com- 
parisons used to determine whether the end of the se- 
quence has been reached.) How does this compare to 
the number of comparisons used by the algorithm in 
Exercise 5? 

Show that the worst-case complexity in terms of compar- 

isons of an algorithm that finds the maximum and mini- 

mum of n elements is at least [3n/2] — 2. 


. Devise an efficient algorithm for finding the second 


largest element in a sequence of n elements and deter- 
mine the worst-case complexity of your algorithm. 


. Devise an algorithm that finds all equal pairs of sums of 


two terms of a sequence of n numbers, and determine the 
worst-case complexity of your algorithm. 


Devise an algorithm that finds the closest pair of integers 
in a sequence of n integers, and determine the worst-case 
complexity of your algorithm. [Hint: Sort the sequence. 
Use the fact that sorting can be done with worst-case time 
complexity O(n logn).] 


The shaker sort (or bidirectional bubble sort) successively 
compares pairs of adjacent elements, exchanging them if they 
are out of order, and alternately passing through the list from 
the beginning to the end and then from the end to the beginning 
until no exchanges are needed. 


11. 


12. 
13. 


14. 


15. 
16. 
17. 
18. 
*19, 
*20. 


Show the steps used by the shaker sort to sort the list 3, 
5, 1, 4, 6, 2. 
Express the shaker sort in pseudocode. 


Show that the shaker sort has O(n”) complexity measured 
in terms of the number of comparisons it uses. 


Explain why the shaker sort is efficient for sorting lists 
that are already in close to the correct order. 


Show that (n logn + n”)3 is O(n). 

Show that 8x? + 12x + 100 log x is O(x?). 

Give a big-O estimate for (x? + x (log x)) - (2° + x3). 
Find a big-O estimate for )7_, j(j + 1). 

Show that n! is not O(2”). 

Show that n” is not O(n!). 
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21. 


22. 


23. 


24. 
* 25, 


* 26. 


* 27, 


28. 


29. 


30. 


Find all pairs of functions of the same order in this 
list of functions: n? + dog ny, n? +n, n2+ log 2” + 1, 
(n+ 1)? — (n— 1)3, and (n + logn)?. 


Find all pairs of functions of the same order in this list of 
functions n2 + 2”, n2 + 2!00 n2 4.22" n2 + n!,n? +3", 
and (n? + 1)?. 


2100 


Find an integer n with n > 2 for whichn e2P, 


Find an integer n with n > 2 for which (log ny” < Jn. 


Arrange the functions n”, (log n)?, n'-0001 (1.0001)”, 


QV 1082" and n(logn)!! in a list so that each function 
is big-O of the next function. [Hint: To determine the 
relative size of some of these functions, take logarithms. ] 


Arrange the function 2100n | Qn grt 92" plogn | 
nlogn log logn, n*/*, n(logn)/”, and n*/3(logn)? in a 
list so that each function is big-O of the next function. 
[Hint: To determine the relative size of some of these 
functions, take logarithms. ] 


Give an example of two increasing functions f(n) and 
g(n) from the set of positive integers to the set of posi- 
tive integers such that neither f(n) is O(g(n)) nor g(n) 
is O(f(n)). 

1 k 


Show that if the denominations of coins are 0, Cs gO; 
where k is a positive integer and c is a positive integer, 
c > 1, the greedy algorithm always produces change us- 
ing the fewest coins possible. 


a) Use pseudocode to specify a brute-force algorithm that 
determines when given as input a sequence of n pos- 
itive integers whether there are two distinct terms of 
the sequence that have as sum a third term. The algo- 
rithm should loop through all triples of terms of the 
sequence, checking whether the sum of the first two 
terms equals the third. 


b) Give a big-O estimate for the complexity of the brute- 
force algorithm from part (a). 


a) Devise a more efficient algorithm for solving the prob- 
lem described in Exercise 29 that first sorts the in- 
put sequence and then checks for each pair of terms 
whether their difference is in the sequence. 


b) Give a big-O estimate for the complexity of this al- 
gorithm. Is it more efficient than the brute-force algo- 
rithm from Exercise 29? 


Suppose we have s men and s women each with their prefer- 
ence lists for the members of the opposite gender, as described 
in the preamble to Exercise 60 in Section 3.1. We say that a 
woman w is a valid partner for a man m if there is some sta- 
ble matching in which they are paired. Similarly, a man m is a 
valid partner for a woman w if there is some stable matching 
in which they are paired. A matching in which each man is as- 
signed his valid partner ranking highest on his preference list 
is called male optimal, and a matching in which each woman 
is assigned her valid partner ranking lowest on her preference 
list is called female pessimal. 


31. 


* 32. 


33. 


* 34, 


Find all valid partners for each man and each woman if 
there are three men m1, m2, and m3 and three women w), 
w2, w3 with these preference rankings of the men for the 
women, from highest to lowest: m1: w3, W1, w2; m2: W3, 
W2, W1; 3: W2, W3, W1; and with these preference rank- 
ings of the women for the men, from highest to lowest: 
W1:™3,M2,M 1, W2: M1, M3, M2; W3:M3,M2, M1. 
Show that the deferred acceptance algorithm given in the 
preamble to Exercise 61 of Section 3.1, always produces 
a male optimal and female pessimal matching. 

Define what it means for a matching to be female optimal 
and for a matching to be male pessimal. 

Show that when woman do the proposing in the deferred 
acceptance algorithm, the matching produced is female 
optimal and male pessimal. 


In Exercises 35 and 36 we consider variations on the problem 
of finding stable matchings of men and women described in 
the preamble to Exercise 61 in Section 3.1. 


* 35. 


* 36. 


In this exercise we consider matching problems where 
there may be different numbers of men and women, so 
that it is impossible to match everyone with a member of 
the opposite gender. 


a) Extend the definition of a stable matching from that 
given in the preamble to Exercise 60 in Section 3.1 
to cover the case where there are unequal numbers of 
men and women. Avoid all cases where a man and a 
woman would prefer each other to their current sit- 
uation, including those involving unmatched people. 
(Assume that an unmatched person prefers a match 
with a member of the opposite gender to remaining 
unmatched.) 

b) Adapt the deferred acceptance algorithm to find sta- 
ble matchings, using the definition of stable matchings 
from part (a), when there are different numbers of men 
and women. 

c) Prove that all matchings produced by the algorithm 
from part (b) are stable, according to the definition 
from part (a). 

In this exercise we consider matching problems where 

some man-woman pairs are not allowed. 

a) Extend the definition of a stable matching to cover 
the situation where there are the same number of men 
and women, but certain pairs of men and women are 
forbidden. Avoid all cases where a man and a woman 
would prefer each other to their current situation, in- 
cluding those involving unmatched people. 

b) Adapt the deferred acceptance algorithm to find stable 
matchings when there are the same number of men 
and women, but certain man-woman pairs are forbid- 
den. Be sure to consider people who are unmatched at 
the end of the algorithm. (Assume that an unmatched 
person prefers a match with a member of the opposite 
gender who is not a forbidden partner to remaining 
unmatched.) 

c) Prove that all matchings produced by the algorithm 
from (b) are stable, according to the definition in part 


(a). 


Exercises 37—40 deal with the problem of scheduling n jobs on 
a single processor. To complete job j, the processor must run 
job j for time t; without interruption. Each job has a dead- 
line d;. If we start job j at time sj, it will be completed at 
time e; = s; + t;. The lateness of the job measures how long 
it finishes after its deadline, that is, the lateness of job j is 
max(0, e; — dj). We wish to devise a greedy algorithm that 
minimizes the maximum lateness of a job among the n jobs. 


37. Suppose we have five jobs with specified required times 
and deadlines: ft) = 25, dj = 50; t2 = 15, dx = 60; £3 
20, d3 = 60; t4 = 5, dq = 55; ts = 10,d5 = 75. Find the 
maximum lateness of any job when the jobs are scheduled 
in this order (and they start at time 0): Job 3, Job 1, Job 4, 
Job 2, Job 5. Answer the same question for the schedule 
Job 5, Job 4, Job 3, Job 1, Job 2. 


38. The slackness of a job requiring time ¢ and with deadline 
d is d —t, the difference between its deadline and the 
time it requires. Find an example that shows that schedul- 
ing jobs by increasing slackness does not always yield a 
schedule with the smallest possible maximum lateness. 


39, Find an example that shows that scheduling jobs in or- 
der of increasing time required does not always yield a 
schedule with the smallest possible maximum lateness. 


*40. Prove that scheduling jobs in order of increasing deadlines 
always produces a schedule that minimizes the maximum 
lateness of a job. [Hint: First show that for a schedule to 
be optimal, jobs must be scheduled with no idle time be- 
tween them and so that no job is scheduled before another 
with an earlier deadline. | 


41. Suppose that we have a knapsack with total capacity of 
W kg. We also have n items where item j has mass w7;. 
The knapsack problem asks for a subset of these n items 
with the largest possible total mass not exceeding W. 


a) Devise a brute-force algorithm for solving the knap- 
sack problem. 


b) Solve the knapsack problem when the capacity of the 
knapsack is 18 kg and there are five items: a 5-kg 


Computer Projects 


Write programs with these inputs and outputs. 


1. Given a list of n integers, find the largest integer in the 
list. 


2. Given alist of n integers, find the first and last occurrences 
of the largest integer in the list. 


3. Given a list of n distinct integers, determine the position 
of an integer in the list using a linear search. 


4. Given an ordered list of n distinct integers, determine the 
position of an integer in the list using a binary search. 


5. Given a list of n integers, sort them using a bubble sort. 
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sleeping bag, an 8-kg tent, a 7-kg food pack, a 4-kg 
container of water, and an 11-kg portable stove. 


In Exercises 42-46 we will study the problem of load balanc- 

ing. The input to the problem is a collection of p processors 

and n jobs, t; is the time required to run job j, jobs run without 
interruption on a single machine until finished, and a proces- 
sor can run only one job at a time. The load L; of processor 

k is the sum over all jobs assigned to processor k of the times 

required to run these jobs. The makespan is the maximum 

load over all the p processors. The load balancing problem 
asks for an assignment of jobs to processors to minimize the 
makespan. 

42. Suppose we have three processors and five jobs requiring 
times t) = 3, fo = 5, BR = 4, t4 = 7, and ts = 8. Solve 
the load balancing problem for this input by finding the 
assignment of the five jobs to the three processors that 
minimizes the makespan. 


43. Suppose that L* is the minimum makespan when p pro- 
cessors are given n jobs, where f; is the time required to 
run job j. 
a) Show that L* > max j=1,2,....n tj. 

1 

b) Show that L* > r Dt tj. 

44. Write out in pseudocode the greedy algorithm that goes 
through the jobs in order and assigns each job to the pro- 
cessor with the smallest load at that point in the algorithm. 


45. Run the algorithm from Exercise 44 on the input given in 
Exercise 42. 

An approximation algorithm for an optimization problem 

produces a solution guaranteed to be close to an optimal so- 

lution. More precisely, suppose that the optimization problem 

asks for an input S that minimizes F(X) where F is some 

function of the input X. If an algorithm always finds an input 

T with F(T) < cF(S) where c is a fixed positive real number, 

the algorithm is called a c-approximation algorithm for the 

problem. 

*46. Prove that the algorithm from Exercise 44 is a 2- 

approximation algorithm for the load balancing problem. 
[Hint: Use both parts of Exercise 43.] 


6. Given a list of n integers, sort them using an insertion 
sort. 


7. Given an integer n, use the greedy algorithm to find the 
change for n cents using quarters, dimes, nickels, and 
pennies. 


8. Given the starting and ending times of n talks, use the 
appropriate greedy algorithm to schedule the most talks 
possible in a single lecture hall. 
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9. Given an ordered list of n integers and an integer x in the 
list, find the number of comparisons used to determine 
the position of x in the list using a linear search and using 
a binary search. 


10. Given a list of integers, determine the number of compar- 
isons used by the bubble sort and by the insertion sort to 
sort this list. 


Computations and Explorations 


Use a computational program or programs you have written to do these exercises. 


1. We know that n’ is O(d") when b and d are positive and determine whether the smallest number of coins was 


numbers with d > 2. Give values of the constants C and 
k such that n? < Cd" whenever x > k for each of these 
sets of values: b = 10,d = 2; b = 20,d = 3; b = 1000, 
d=7. 

. Compute the change for different values of n with coins 
of different denominations using the greedy algorithm 


Writing Projects 


Respond to these with essays using outside sources. 


. Examine the history of the word algorithm and describe 
the use of this word in early writings. 

. Look up Bachmann’s original introduction of big-O no- 
tation. Explain how he and others have used this notation. 
. Explain how sorting algorithms can be classified into a 
taxonomy based on the underlying principle on which 
they are based. 

. Describe the radix sort algorithm. 

. Describe the historic trends in how quickly processors can 
perform operations and use these trends to estimate how 
quickly processors will be able to perform operations in 
the next twenty years. 

. Develop a detailed list of algorithmic paradigms and pro- 
vide examples using each of these paradigms. 


10. 
11. 


used. Can you find conditions so that the greedy algorithm 
is guaranteed to use the fewest coins possible? 


. Using a generator of random orderings of the integers 


1,2,...,”, find the number of comparisons used by 
the bubble sort, insertion sort, binary insertion sort, and 
selection sort to sort these integers. 


. Explain what the Turing Award is and describe the criteria 


used to select winners. List six past winners of the award 
and why they received the award. 


. Describe what is meant by a parallel algorithm. Explain 


how the pseudocode used in this book can be extended to 
handle parallel algorithms. 


. Explain how the complexity of parallel algorithms can be 


measured. Give some examples to illustrate this concept, 
showing how a parallel algorithm can work more quickly 
than one that does not operate in parallel. 


Describe six different NP-complete problems. 


Demonstrate how one of the many different NP-complete 
problems can be reduced to the satisfiability problem. 
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Number Theory and Cryptography 


he part of mathematics devoted to the study of the set of integers and their properties is 

known as number theory. In this chapter we will develop some of the important concepts 
of number theory including many of those used in computer science. As we develop number 
theory, we will use the proof methods developed in Chapter | to prove many theorems. 

We will first introduce the notion of divisibility of integers, which we use to introduce 
modular, or clock, arithmetic. Modular arithmetic operates with the remainders of integers 
when they are divided by a fixed positive integer, called the modulus. We will prove many 
important results about modular arithmetic which we will use extensively in this chapter. 

Integers can be represented with any positive integer b greater than | as a base. In this 
chapter we discuss base b representations of integers and give an algorithm for finding them. 
In particular, we will discuss binary, octal, and hexadecimal (base 2, 8, and 16) representations. 
We will describe algorithms for carrying out arithmetic using these representations and study 
their complexity. These algorithms were the first procedures called algorithms. 

We will discuss prime numbers, the positive integers that have only | and themselves 
as positive divisors. We will prove that there are infinitely many primes; the proof we give is 
considered to be one of the most beautiful proofs in mathematics. We will discuss the distribution 
of primes and many famous open questions concerning primes. We will introduce the concept of 
greatest common divisors and study the Euclidean algorithm for computing them. This algorithm 
was first described thousands of years ago. We will introduce the fundamental theorem of 
arithmetic, a key result which tells us that every positive integer has a unique factorization into 
primes. 

We will explain how to solve linear congruences, as well as systems of linear congruences, 
which we solve using the famous Chinese remainder theorem. We will introduce the notion of 
pseudoprimes, which are composite integers masquerading as primes, and show how this notion 
can help us rapidly generate prime numbers. 

This chapter introduces several important applications of number theory. In particular, we 
will use number theory to generate pseudorandom numbers, to assign memory locations to 
computer files, and to find check digits used to detect errors in various kinds of identification 
numbers. We also introduce the subject of cryptography. Number theory plays an essentially 
role both in classical cryptography, first used thousands of years ago, and modern cryptography, 
which plays an essential role in electronic communication. We will show how the ideas we 
develop can be used in cryptographical protocols, introducing protocols for sharing keys and for 
sending signed messages. Number theory, once considered the purest of subjects, has become 
an essential tool in providing computer and Internet security. 


Divisibility and Modular Arithmetic 


Introduction 


The ideas that we will develop in this section are based on the notion of divisibility. Division of an 
integer by a positive integer produces a quotient and a remainder. Working with these remainders 
leads to modular arithmetic, which plays an important role in mathematics and which is used 
throughout computer science. We will discuss some important applications of modular arithmetic 
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THEOREM 1 


later in this chapter, including generating pseudorandom numbers, assigning computer memory 
locations to files, constructing check digits, and encrypting messages. 


Division 


When one integer is divided by a second nonzero integer, the quotient may or may not be 
an integer. For example, 12/3 = 4 is an integer, whereas 11/4 = 2.75 is not. This leads to 
Definition 1. 


If a and b are integers with a # 0, we say that a divides b if there is an integer c such that 
b = ac, or equivalently, if B is an integer. When a divides b we say that a is a factor or divisor 
of b, and that b is a multiple of a. The notation a | b denotes that a divides b. We write a / b 
when a does not divide b. 


Remark: We can express a | b using quantifiers as c(ac = b), where the universe of discourse 
is the set of integers. 

In Figure | a number line indicates which integers are divisible by the positive integer d. 
Determine whether 3 | 7 and whether 3 | 12. 


Solution: We see that 3 / 7, because 7/3 is not an integer. On the other hand, 3 | 12 because 
12/3 = 4. < 


Let n and d be positive integers. How many positive integers not exceeding n are divisible by d? 


Solution: The positive integers divisible by d are all the integers of the form dk, where k is 
a positive integer. Hence, the number of positive integers divisible by d that do not exceed n 
equals the number of integers k with 0 < dk <n, or with 0 < k < n/d. Therefore, there are 
[n/d| positive integers not exceeding n that are divisible by d. < 


Some of the basic properties of divisibility of integers are given in Theorem 1. 


Let a, b, and c be integers, where a 4 0. Then 


(i) ifa | banda |c,thena | (b+ c); 
(ii) if a | b, then a | bc for all integers c; 
(iii) ifa | bandb|c, thena | c. 


Proof: We will give a direct proof of (i). Suppose that a | b and a | c. Then, from the definition 
of divisibility, it follows that there are integers s and ¢ with b = as and c = at. Hence, 


b+c=as+at=a(s +t). 


FIGURE 1 Integers Divisible by the Positive Integer d. 


COROLLARY 1 


THEOREM 2 


DEFINITION 2 
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Therefore, a divides b + c. This establishes part (i) of the theorem. The proofs of parts (ii) and 
(iii) are left as Exercises 3 and 4. < 


Theorem | has this useful consequence. 


If a, b, and c are integers, where a 4 0, such that a | b anda | c, thena | mb + nc whenever 
m and n are integers. 


Proof: We will give a direct proof. By part (ii) of Theorem | we see that a | mb and a | ne 
whenever m and n are integers. By part (i) of Theorem | it follows that a | mb + nc. 4 
The Division Algorithm 


When an integer is divided by a positive integer, there is a quotient and a remainder, as the 
division algorithm shows. 


THE DIVISION ALGORITHM Let a be an integer and d a positive integer. Then there 
are unique integers g andr, withO <r <d,suchthata =dq +r. 


We defer the proof of the division algorithm to Section 5.2. (See Example 5 and 
Exercise 37.) 


Remark: Theorem 2 is not really an algorithm. (Why not?) Nevertheless, we use its traditional 
name. 


In the equality given in the division algorithm, d is called the divisor, a is called the dividend, 
q 1s called the quotient, and r is called the remainder. This notation is used to express the 
quotient and remainder: 
q=adivd, r=amodd. 
Remark: Note that both a div d and a mod d for a fixed d are functions on the set of inte- 
gers. Furthermore, when a is an integer and d is a positive integer, we have a div d = |a/d| 
and a mod d = a — d. (See exercise 18.) 
Examples 3 and 4 illustrate the division algorithm. 


What are the quotient and remainder when 101 is divided by 11? 


Solution: We have 
101 = 11-942. 


Hence, the quotient when 101 is divided by 11 is 9 = 101 div 11, and the remainder is 
2 = 101 mod 11. < 
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EXAMPLE 4 What are the quotient and remainder when —11 is divided by 3? 
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DEFINITION 3 


Solution: We have 
—11 = 3(—4) +1. 


Hence, the quotient when —11 is divided by 3 is —4 = —11 div 3, and the remainder is 
1 = —11mod 3. 

Note that the remainder cannot be negative. Consequently, the remainder is not —2, even 
though 


—11 = 3(-3) - 2, 
because r = —2 does not satisfy 0 < r < 3. < 


Note that the integer a is divisible by the integer d if and only if the remainder is zero 
when a is divided by d. 


Remark: A programming language may have one, or possibly two, operators for modular arith- 
metic, denoted by mod (in BASIC, Maple, Mathematica, EXCEL, and SQL), % (in C, C++, Java, 
and Python), rem (in Ada and Lisp), or something else. Be careful when using them, because 
for a < 0, some of these operators return a — m[a/m] instead of amodm = a — m|a/m| (as 
shown in Exercise 18). Also, unlike a mod m, some of these operators are defined when m < 0, 
and even when m = 0. 


Modular Arithmetic 


In some situations we care only about the remainder of an integer when it is divided by some 
specified positive integer. For instance, when we ask what time it will be (on a 24-hour clock) 50 
hours from now, we care only about the remainder when 50 plus the current hour is divided by 24. 
Because we are often interested only in remainders, we have special notations for them. We have 
already introduced the notation a mod m to represent the remainder when an integer a is divided 
by the positive integer m. We now introduce a different, but related, notation that indicates that 
two integers have the same remainder when they are divided by the positive integer m. 


If a and b are integers and m is a positive integer, then a is congruent to b modulo m if 
m divides a — b. We use the notation a = b (mod m) to indicate that a is congruent to 
b modulo m. We say that a = b (mod m) is a congruence and that m is its modulus (plural 
moduli). If a and b are not congruent modulo m, we write a # b (mod m). 


Although both notations a = b (mod m) and a mod m = b include “mod,” they represent 
fundamentally different concepts. The first represents a relation on the set of integers, whereas 
the second represents a function. However, the relation a = b (mod m) and the mod m function 
are closely related, as described in Theorem 3. 


THEOREM 3 


EXAMPLE 5 


THEOREM 4 
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Let a and b be integers, and let m be a positive integer. Then a = b (mod m) if and only 
if a mod m = b mod m. 


The proof of Theorem 3 is left as Exercises 15 and 16. Recall that a mod m and b mod m are 
the remainders when a and b are divided by m, respectively. Consequently, Theorem 3 also says 
that a = b (mod m) if and only if a and b have the same remainder when divided by m. 


Determine whether 17 is congruent to 5 modulo 6 and whether 24 and 14 are congruent modulo 6. 


Solution: Because 6 divides 17 — 5 = 12, we see that 17 = 5 (mod 6). However, because 
24 — 14 = 10 is not divisible by 6, we see that 24 4 14 (mod 6). < 


The great German mathematician Karl Friedrich Gauss developed the concept of congru- 
ences at the end of the eighteenth century. The notion of congruences has played an important 
role in the development of number theory. 

Theorem 4 provides a useful way to work with congruences. 


Let m be a positive integer. The integers a and b are congruent modulo m if and only if there 
is an integer k such thata = b+ km. 


Proof: If a=b (mod m), by the definition of congruence (Definition 3), we know that 
m | (a — b). This means that there is an integer k such that a — b = km, so thata = b+km. 
Conversely, if there is an integer k such that a = b + km, then km = a — b. Hence, m divides 
a — b, so that a = b (mod m). <j 


The set of all integers congruent to an integer a modulo m is called the congruence class 
of a modulo m. In Chapter 9 we will show that there are m pairwise disjoint equivalence classes 
modulo m and that the union of these equivalence classes is the set of integers. 

Theorem 5 shows that additions and multiplications preserve congruences. 


KARL FRIEDRICH GAUSS (1777-1855) Karl Friedrich Gauss, the son of a bricklayer, was a child prodigy. 
He demonstrated his potential at the age of 10, when he quickly solved a problem assigned by a teacher to keep 
the class busy. The teacher asked the students to find the sum of the first 100 positive integers. Gauss realized 
that this sum could be found by forming 50 pairs, each with the sum 101: 1+ 100,24 99,...,50+4 51. 
This brilliance attracted the sponsorship of patrons, including Duke Ferdinand of Brunswick, who made it 
possible for Gauss to attend Caroline College and the University of Gottingen. While a student, he invented 
the method of least squares, which is used to estimate the most likely value of a variable from experimental 
results. In 1796 Gauss made a fundamental discovery in geometry, advancing a subject that had not advanced 
since ancient times. He showed that a 17-sided regular polygon could be drawn using just a ruler and compass. 


In 1799 Gauss presented the first rigorous proof of the fundamental theorem of algebra, which states that a polynomial of 
degree n has exactly n roots (counting multiplicities). Gauss achieved worldwide fame when he successfully calculated the orbit of 
the first asteroid discovered, Ceres, using scanty data. 

Gauss was called the Prince of Mathematics by his contemporary mathematicians. Although Gauss is noted for his many 
discoveries in geometry, algebra, analysis, astronomy, and physics, he had a special interest in number theory, which can be seen 
from his statement “Mathematics is the queen of the sciences, and the theory of numbers is the queen of mathematics.” Gauss laid 
the foundations for modern number theory with the publication of his book Disquisitiones Arithmeticae in 1801. 
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of a congruence 

by the same number! 


COROLLARY 2 


Let m be a positive integer. If a = b (mod m) and c = d (mod m), then 


a+c=b+d (modm) and ac = bd (mod m). 


Proof: We use a direct proof. Because a = b (mod m) and c = d (mod m), by Theorem 4 there 
are integers s and t with b = a+sm andd =c + tm. Hence, 


b+dz=(a+sm)+(c+tm) = (a+c)+m(s +f) 
and 

bd =(a+sm)(c+tm) =ac+m(at +cs + stm). 
Hence, 


a+c=b+d (modm) and ac = bd (mod m). <q 


Because 7 = 2 (mod 5) and 11 = 1 (mod 5), it follows from Theorem 5 that 
18=74+11=2+1=3 (mod5) 


and that 


77 =7-11=2-1=2(mod5). < 


We must be careful working with congruences. Some properties we may expect to be true 
are not valid. For example, if ac = bc (mod m), the congruence a = b (mod m) may be false. 
Similarly, if a = b (mod m) and c = d (mod m), the congruence a® = b4 (mod m) may be 
false. (See Exercise 37.) 

Corollary 2 shows how to find the values of the mod m function at the sum and product of 
two integers using the values of this function at each of these integers. We will use this result in 
Section 5.4, 


Let m be a positive integer and let a and b be integers. Then 
(a + b) mod m = ((amod m) + (bmodm)) mod m 
and 


abmod m = ((amod m)(b mod m)) mod m. 


Proof: By the definitions of modm and of congruence modulo m, we know that a = 
(a mod m) (modm) and b = (bmodm) (modm). Hence, Theorem 5 tells us that 


a+b=(amodm) + (bmodm) (modm) 
and 
ab = (amod m)(bmod m) (mod m). 


The equalities in this corollary follow from these last two congruences by Theorem 3. | 


EXAMPLE 7 
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Arithmetic Modulo m 


We can define arithmetic operations on Z,,, the set of nonnegative integers less than m, that is, 
the set {0, 1,...,m — 1}. In particular, we define addition of these integers, denoted by +,, by 


a+mb=(a+b)modm, 


where the addition on the right-hand side of this equation is the ordinary addition of integers, 
and we define multiplication of these integers, denoted by -, by 


a+m b= (a-b)modm, 


where the multiplication on the right-hand side of this equation is the ordinary multiplication of 
integers. The operations +,, and -,, are called addition and multiplication modulo m and when 
we use these operations, we are said to be doing arithmetic modulo m. 


Use the definition of addition and multiplication in Z,, to find 7 +1; 9 and 7 -1; 9. 
Solution: Using the definition of addition modulo 11, we find that 
7+119=(7+9) mod 11 = 16 mod 11 =5, 
and 
7-119 = (7-9)mod 11 = 63 mod 11 = 8. 
Hence 7 +); 9 =5and7-;;9=8. < 
The operations +,, and -,, satisfy many of the same properties of ordinary addition and 
multiplication of integers. In particular, they satisfy these properties: 


Closure If a and b belong to Z,,, then ad +, b and a -» b belong to Zp. 


Associativity If a, b, and c belong to Zm, then (4+mb)+mc=a4+m(b+mc) and 
(amb) +m C =A-m (bm C). 


Commutativity If a and b belong to Z,,, thena +, b=b+maandda-_b=b-m a. 


Identity elements The elements 0 and | are identity elements for addition and multiplication 
modulo m, respectively. That is, if a belongs to Z,,, then a +,0=0+ ,)a=aanda-,1= 
l-ma=a. 


Additive inverses If a 4 0 belongs to Z,,,, then m — a is an additive inverse of a modulo m and 
0 is its own additive inverse. That is a + (m — a) = 0 and0+,, 0 = 0. 


Distributivity If a, b, and c belong to Z,, then a +m (b+ Cc) = (4 +m b) +m (a-m cc) and 
(a4 +m b) +m € = (4+m C) +m (b-m ce). 


These properties follow from the properties we have developed for congruences and remainders 
modulo m, together with the properties of integers; we leave their proofs as Exercises 42-44. 
Note that we have listed the property that every element of Z,, has an additive inverse, but no 
analogous property for multiplicative inverses has been included. This is because multiplicative 
inverses do not always exists modulo m. For instance, there is no multiplicative inverse of 2 
modulo 6, as the reader can verify. We will return to the question of when an integer has a 
multiplicative inverse modulo m later in this chapter. 
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Remark: Because Z, with the operations of addition and multiplication modulo m satisfies 
the properties listed, Z,, with modular addition is said to be a commutative group and Z,, 
with both of these operations is said to be a commutative ring. Note that the set of integers 
with ordinary addition and multiplication also forms a commutative ring. Groups and rings are 
studied in courses that cover abstract algebra. 


Remark: In Exercise 30, and in later sections, we will use the notations + and - for +,, and - 
without the subscript m on the symbol for the operator whenever we work with Z. 


Exercises 


1. 


Does 17 divide each of these numbers? 
a) 68 b) 84 c) 357 d) 1001 


. Prove that if a is an integer other than 0, then 


a) 1 divides a. b) a divides 0. 


. Prove that part (ii) of Theorem | is true. 
. Prove that part (iii) of Theorem | is true. 
. Show that if a | b and b | a, where a and b are integers, 


thena = bora=-—b. 


. Show that if a, b,c, and d are integers, where a 4 0, such 


that a | c and b | d, then ab | cd. 


. Show that if a, b, and c are integers, where a ~ 0 and 


c # 0, such that ac | bc, then a | b. 


. Prove or disprove that if a | bc, where a, b, and c are pos- 


itive integers anda # 0, thena|bora|c. 


. What are the quotient and remainder when 


a) 19 is divided by 7? 
b) —111 is divided by 11? 
c) 789 is divided by 23? 
d) 1001 is divided by 13? 
e) (is divided by 19? 

f) 3 is divided by 5? 

g) —1 is divided by 3? 

h) 4 is divided by 1? 


10. What are the quotient and remainder when 


11. 


a) 44 is divided by 8? 

b) 777 is divided by 21? 

c) —123 is divided by 19? 

d) —1 is divided by 23? 

e) —2002 is divided by 87? 

f) Ois divided by 17? 

g) 1,234,567 is divided by 1001? 
h) —100 is divided by 101? 

What time does a 12-hour clock read 
a) 80 hours after it reads 11:00? 
b) 40 hours before it reads 12:00? 
c) 100 hours after it reads 6:00? 


12. What time does a 24-hour clock read 


a) 100 hours after it reads 2:00? 
b) 45 hours before it reads 12:00? 
c) 168 hours after it reads 19:00? 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


Suppose that a and b are integers, a = 4 (mod 13), and 
b = 9 (mod 13). Find the integer c withO < c < 12 such 
that 


a) c = 9a (mod 13). 

b) c = 11b (mod 13). 

c) c = a+b (mod 13). 
d) c = 2a+ 3b (mod 13). 
e) c = a2 +b? (mod 13). 
f) c = a —b (mod 13). 


Suppose that a and b are integers,a = 11 (mod 19), and 
b = 3 (mod 19). Find the integer c withO < c < 18 such 
that 


a) c = 13a (mod 19). 
b) c = 8b (mod 19). 

c) c = a—b(mod 19). 

d) c = 7a + 3b (mod 19). 
e) c = 2a” + 3b? (mod 19). 


f) c = a + 4b? (mod 19). 

Let m be a positive integer. Show that a = b (mod m) if 
a mod m = b mod m. 

Let m be a positive integer. Show that a mod m = 
b mod m if a = b(modm). 

Show that if n and k are positive integers, then [n/k] = 
L(n —1)/k] +1. 

Show that if a is an integer and d is an inte- 
ger greater than 1, then the quotient and remain- 
der obtained when a is divided by d are |a/d]| and 
a — d|a/d|, respectively. 

Find a formula for the integer with smallest absolute value 
that is congruent to an integer a modulo m, where m is a 
positive integer. 

Evaluate these quantities. 


a) —17 mod 2 b) 144 mod 7 

c) —101 mod 13 d) 199 mod 19 
Evaluate these quantities. 

a) 13 mod 3 b) —97 mod 11 
c) 155 mod 19 d) —221 mod 23 


Find a div m and a mod m when 
a) a=—-ll1ll,m=99. 

b) a = —9999, m = 101. 

c) a= 10299, m = 999. 

d) a = 123456, m = 1001. 


23. 


24. 


25. 


26. 
27. 


28. 


29. 


30. 


31. 


32. 


33. 


Find a div m and a mod m when 

a) a= 228,m = 119. 

b) a = 9009, m = 223. 

c) a= —10101, m = 333. 

d) a = —765432, m = 38271. 

Find the integer a such that 

a) a = 43 (mod 23) and —22 <a <0. 

b) a = 17 (mod 29) and —14 <a < 14. 

c) a = —11 (mod 21) and 90 < a < 110. 

Find the integer a such that 

a) a = —15 (mod 27) and —26 <a < 0. 

b) a = 24(mod 31) and -15 < a < 15. 

c) a = 99 (mod 41) and 100 < a < 140. 

List five integers that are congruent to 4 modulo 12. 
List all integers between — 100 and 100 that are congruent 
to —1 modulo 25. 

Decide whether each of these integers is congruent to 
3 modulo 7. 

a) 37 b) 66 

c) -17 d) —67 

Decide whether each of these integers is congruent to 
5 modulo 17. 

a) 80 

c) —29 

Find each of these values. 
a) (177mod 31 + 270mod 31) mod 31 
b) (177 mod 31 - 270 mod 31) mod 31 
Find each of these values. 

a) (—133mod 23 + 261 mod 23) mod 23 
b) (457 mod 23 - 182 mod 23) mod 23 
Find each of these values. 

a) (197 mod 41) mod 9 

b) (323 mod 13)? mod 11 

c) (7? mod 23)? mod 31 

d) (217 mod 15)? mod 22 

Find each of these values. 

a) (99° mod 32)? mod 15 

b) (34+ mod 17)? mod 11 

c) (19° mod 23)? mod 31 

d) (893 mod 79)* mod 26 


b) 103 
d) —122 


cS 


34. 


35. 


36. 


37. 


38. 
39. 


40. 


41. 


42. 


43. 


44. 


45. 


46. 


47. 
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Show that if a = b (mod m) and c = d (mod m), where 
a, b, c, d, and m are integers with m > 2, thena-—c= 
b—d(modm). 

Show that if n | m, where n and m are integers greater 

than 1, and if a = b (mod m), where a and b are integers, 

then a = b (mod n). 

Show that if a, b, c, and m are integers such that m > 2, 

c > 0, and a = b (mod m), then ac = bc (mod mc). 

Find counterexamples to each of these statements about 

congruences. 

a) Ifac = bc (mod m), where a, b, c, and m are integers 
with m > 2, then a = b (mod m). 

b) If a=b (mod m) and c=d (mod m), where 
a, b, c, d, and m are integers with c and d positive 
and m > 2, then a® = b¢ (mod m). 

Show that if 1 is an integer then n? = Oor1(mod 4). 

Use Exercise 38 to show that if m is a positive integer of 

the form 4k + 3 for some nonnegative integer k, then m 

is not the sum of the squares of two integers. 

Prove that if n is an odd positive integer, then n? = 

1 (mod 8). 

Show that if a, b, k, and m are integers such that k > 1, 

m > 2, anda = b(modm), then ak = b*(mod m). 

Show that Z,, with addition modulo m, where m > 2 is 

an integer, satisfies the closure, associative, and commu- 

tative properties, 0 is an additive identity, and for every 

nonzero a € Z», m — a is an inverse of a modulo m. 

Show that Z,, with multiplication modulo m, where 

m > 2 is an integer, satisfies the closure, associative, and 

commutativity properties, and 1| is a multiplicative iden- 

tity. 

Show that the distributive property of multiplication over 

addition holds for Z,,, where m > 2 is an integer. 

Write out the addition and multiplication tables for Zs 

(where by addition and multiplication we mean +5 

and -s). 

Write out the addition and multiplication tables for Ze 

(where by addition and multiplication we mean +6 

and -6). 

Determine whether each of the functions f(a) = a divd 

and g(a) = a mod d, where d is a fixed positive integer, 

from the set of integers to the set of integers, is one-to-one, 
and determine whether each of these functions is onto. 


Integer Representations and Algorithms 


Introduction 


Integers can be expressed using any integer greater than one as a base, as we will show in 
this section. Although we commonly use decimal (base 10), representations, binary (base 2), 
octal (base 8), and hexadecimal (base 16) representations are often used, especially in computer 
science. Given a base b and an integer n, we will show how to construct the base b representation 
of this integer. We will also explain how to quickly covert between binary and octal and between 


binary and hexadecimal notations. 
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THEOREM 1 


EXAMPLE 1 


As mentioned in Section 3.1, the term algorithm originally referred to procedures for per- 
forming arithmetic operations using the decimal representations of integers. These algorithms, 
adapted for use with binary representations, are the basis for computer arithmetic. They provide 
good illustrations of the concept of an algorithm and the complexity of algorithms. For these 
reasons, they will be discussed in this section. 

We will also introduce an algorithm for finding a div d and a mod d where a and d are 
integers withd > 1. Finally, we will describe an efficient algorithm for modular exponentiation, 
which is a particularly important algorithm for cryptography, as we will see in Section 4.6. 


Representations of Integers 


In everyday life we use decimal notation to express integers. For example, 965 is used to denote 
9-107 +6-10+5. However, it is often convenient to use bases other than 10. In particular, 
computers usually use binary notation (with 2 as the base) when carrying out arithmetic, and 
octal (base 8) or hexadecimal (base 16) notation when expressing characters, such as letters or 
digits. In fact, we can use any integer greater than | as the base when expressing integers. This 
is stated in Theorem 1. 


Let b be an integer greater than 1. Then if 7 is a positive integer, it can be expressed uniquely 
in the form 


n = axb* + ay_1be 1 4+.---+a;b+ a0, 


where k is a nonnegative integer, ap, a1, ..., ax are nonnegative integers less than b, and 


Ge == (0), 


A proof of this theorem can be constructed using mathematical induction, a proof method that 
is discussed in Section 5.1. It can also be found in [Ro10]. The representation of n given in 
Theorem | is called the base b expansion of n. The base b expansion of n is denoted by 
(azax—| . . .@1a0)p- For instance, (245) represents 2 - 874+4-845= 165. Typically, the sub- 
script 10 is omitted for base 10 expansions of integers because base 10, or decimal expansions, 
are commonly used to represent integers. 


BINARY EXPANSIONS Choosing 2 as the base gives binary expansions of integers. In 
binary notation each digit is either a 0 or a 1. In other words, the binary expansion of an 
integer is just a bit string. Binary expansions (and related expansions that are variants of binary 
expansions) are used by computers to represent and do arithmetic with integers. 


What is the decimal expansion of the integer that has (1 0101 1111)2 as its binary expansion? 


Solution: We have 


(10101 1111)5 =1+2" + 0.2" + 1.2" 4 0-27 41.9% 
Ae fa Be <9 22! 1S 51, | 


OCTAL AND HEXADECIMAL EXPANSIONS Among the most important bases in com- 
puter science are base 2, base 8, and base 16. Base 8 expansions are called octal expansions and 
base 16 expansions are hexadecimal expansions. 


EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 
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What is the decimal expansion of the number with octal expansion (7016)? 
Solution: Using the definition of a base b expansion with b = 8 tells us that 
(7016)g =7-8°+0-8° + 1-846 = 3598. < 
Sixteen different digits are required for hexadecimal expansions. Usually, the hexadecimal 
digits used are 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, and F, where the letters A through F 
represent the digits corresponding to the numbers 10 through 15 (in decimal notation). 


What is the decimal expansion of the number with hexadecimal expansion (2AEO0B) 16? 
Solution: Using the definition of a base b expansion with b = 16 tells us that 
(2AEOB) 16 = 2- 164+ 10- 16° + 14-167 + 0-164 11 = 175627. < 
Each hexadecimal digit can be represented using four bits. For instance, we see that 
(1110 0101)2 = (E5) {6 because (1110)2 = (E)16 and (0101)2 = (5)16. Bytes, which are bit 


strings of length eight, can be represented by two hexadecimal digits. 


BASE CONVERSION We will now describe an algorithm for constructing the base b expan- 
sion of an integer n. First, divide n by b to obtain a quotient and remainder, that is, 


n = bqo+ ao, 0 <ao <b. 


The remainder, ao, is the rightmost digit in the base b expansion of n. Next, divide go by b to 
obtain 


go=bqait+ai, Ox<a <b. 
We see that a; is the second digit from the right in the base b expansion of n. Continue this 
process, successively dividing the quotients by b, obtaining additional base b digits as the 


remainders. This process terminates when we obtain a quotient equal to zero. It produces the 
base b digits of n from the right to the left. 


Find the octal expansion of (12345) 19. 

Solution: First, divide 12345 by 8 to obtain 
12345 = 8 - 1543 + 1. 

Successively dividing quotients by 8 gives 


1543 = 8- 192+ 7, 


192 = 8 - 24+ 0, 
24=8-3+0, 
3=8-04+3. 


The successive remainders that we have found, 1, 7, 0, 0, and 3, are the digits from the right to 
the left of 12345 in base 8. Hence, 


(12345) 19 = (30071)g. < 
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EXAMPLE 5 Find the hexadecimal expansion of (177130)10. 


Solution: First divide 177130 by 16 to obtain 
177130 = 16- 11070 + 10. 
Successively dividing quotients by 16 gives 
11070 = 16-691 + 14, 
691 = 16-4343, 
43 = 16-2+ 11, 


2= 16-042. 


The successive remainders that we have found, 10, 14, 3, 11, 2, give us the digits from the right 
to the left of 177130 in the hexadecimal (base 16) expansion of (177130) 0. It follows that 


(177130) 19 = (2B3EA) 16. 
(Recall that the integers 10, 11, and 14 correspond to the hexadecimal digits A, B, and E, 


respectively.) < 


EXAMPLE 6 Find the binary expansion of (241) 9. 


Solution: First divide 241 by 2 to obtain 
241 =2-120+1. 


Successively dividing quotients by 2 gives 


120 = 2-60+0, 
60 = 2-30+0, 
30 =2-15+0, 
1I5=2-7+1, 
7=2-3+4+41, 
3=2-1+1, 
1=2-0+1. 


The successive remainders that we have found, 1, 0, 0, 0, 1, 1, 1, 1, are the digits from the right 
to the left in the binary (base 2) expansion of (241) 19. Hence, 


(241) 9 = (1111 0001)>. < 


The pseudocode given in Algorithm | finds the base b expansion (az—1 ...a1a0)p of the 
integer n. 
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TABLE 1 Hexadecimal, Octal, and Binary Representation of the Integers 0 through 15. 
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EXAMPLE 7 


ALGORITHM 1 Constructing Base b Expansions. 


procedure base b expansion(n, b: positive integers with b > 1) 


q:i=n 
k:=0 
while g 4 0 
a, ‘= q mod b 
q := q divb 
k:=k+1 
return (ag_1,..., 41,40) {(ax_-1 ... 1d) p is the base b expansion of n} 


In Algorithm 1, g represents the quotient obtained by successive divisions by b, starting with 
q =n. The digits in the base b expansion are the remainders of these divisions and are given by 
q mod b. The algorithm terminates when a quotient g = 0 is reached. 


Remark: Note that Algorithm 1 can be thought of as a greedy algorithm, as the base b digits 
are taken as large as possible in each step. 


CONVERSION BETWEEN BINARY, OCTAL, AND HEXADECIMAL EXPANSIONS 
Conversion between binary and octal and between binary and hexadecimal expansions is ex- 
tremely easy because each octal digit corresponds to a block of three binary digits and each 
hexadecimal digit corresponds to a block of four binary digits, with these correspondences 
shown in Table 1 without initial Os shown. (We leave it as Exercises 13-16 to show that this is 
the case.) This conversion is illustrated in Example 7. 


Find the octal and hexadecimal expansions of (11 1110 1011 1100)2 and the binary expansions 
of (765)g and (A8D)j6. 


Solution: To convert (11 1110 1011 1100) into octal notation we group the binary dig- 
its into blocks of three, adding initial zeros at the start of the leftmost block if necessary. 
These blocks, from left to right, are 011, 111, 010, 111, and 100, corresponding to 3, 7, 2, 7, 
and 4, respectively. Consequently, (11 1110 1011 1100)2 = (37274). To convert (11 1110 1011 
1100), into hexadecimal notation we group the binary digits into blocks of four, adding initial 
zeros at the start of the leftmost block if necessary. These blocks, from left to right, are 0011, 
1110, 1011, and 1100, corresponding to the hexadecimal digits 3, E, B, and C, respectively. 
Consequently, (11 1110 1011 1100)2 = (3EBC) 6. 

To convert (765)g into binary notation, we replace each octal digit by a block of three binary 
digits. These blocks are 111, 110, and 101. Hence, (765)g = (1 1111 0101)2. To convert (A8D)j16 
into binary notation, we replace each hexadecimal digit by a block of four binary digits. These 
blocks are 1010, 1000, and 1101. Hence, (A8D)j6 = (1010 1000 1101)2. 
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EXAMPLE 8 


Algorithms for Integer Operations 


The algorithms for performing operations with integers using their binary expansions are ex- 
tremely important in computer arithmetic. We will describe algorithms for the addition and the 
multiplication of two integers expressed in binary notation. We will also analyze the compu- 
tational complexity of these algorithms, in terms of the actual number of bit operations used. 
Throughout this discussion, suppose that the binary expansions of a and b are 


a = (dn—14n—2-..4140)2, b = (bn—1bn—2...b1b0)2, 
so that a and b each have n bits (putting bits equal to 0 at the beginning of one of these expansions 
if necessary). 


We will measure the complexity of algorithms for integer arithmetic in terms of the number 
of bits in these numbers. 


ADDITION ALGORITHM Consider the problem of adding two integers in binary notation. 
A procedure to perform addition can be based on the usual method for adding numbers with 
pencil and paper. This method proceeds by adding pairs of binary digits together with carries, 
when they occur, to compute the sum of two integers. This procedure will now be specified 
in detail. 

To add a and J, first add their rightmost bits. This gives 


ay + bo = co -2+ 50, 


where so is the rightmost bit in the binary expansion of a + b and co is the carry, which is either 
0 or 1. Then add the next pair of bits and the carry, 


aytbht+co=c,-2+51, 


where s, is the next bit (from the right) in the binary expansion of a + b, and c, is the carry. 
Continue this process, adding the corresponding bits in the two binary expansions and the carry, 
to determine the next bit from the right in the binary expansion of a + b. At the last stage, add 
An—1, bn—1, and Cp—2 to obtain cy—; - 2 + s,—1. The leading bit of the sum is s, = cp,— 1. This 
procedure produces the binary expansion of the sum, namely, a + b = (5n5y—15p—2 -. . 5150)2- 


Add a = (1110)> and b = (1011)>. 


Solution: Following the procedure specified in the algorithm, first note that 
ag+bp5 =0+1=0-2+1, 

so that co = O and sg = 1. Then, because 
a+b+ecp9=14+14+0=1-240, 

it follows that cj = | and s; = 0. Continuing, 


agtby+ce,=1+0+1=1-2+0, 


yi ig 
1110 
+1011 


11001 
FIGURE 1 


Adding (1110), 
and (1011). 
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so that cz = | and s2 = 0. Finally, because 
a3+b3+co=14+14+1=1-2+4+1, 


follows that cz = 1 and s3 = 1. This means that sy = cz = 1. Therefore,s = a+ b = (11001)o. 
This addition is displayed in Figure 1, where carries are shown in blue. < 


The algorithm for addition can be described using pseudocode as follows. 


ALGORITHM 2 Addition of Integers. 


procedure add(a, b: positive integers) 
{the binary expansions of a and b are (dn—1dn—2...a1a0)2 
and (b,-1by-2 ...b,bo)2, respectively} 
c:=0 
for j :=Oton—1 
d:= ((aj + bj +.c)/2] 
Sj = aj +bj +c—2d 
cHd 
Seite 
return (so, 5}, ...,5,) {the binary expansion of the sum is (5,5,—1 ...89)2} 


Next, the number of additions of bits used by Algorithm 2 will be analyzed. 


How many additions of bits are required to use Algorithm 2 to add two integers with n bits (or 
less) in their binary representations? 


Solution: Two integers are added by successively adding pairs of bits and, when it occurs, a carry. 
Adding each pair of bits and the carry requires two additions of bits. Thus, the total number of 
additions of bits used is less than twice the number of bits in the expansion. Hence, the number 
of additions of bits used by Algorithm 2 to add two n-bit integers is O(n). 


MULTIPLICATION ALGORITHM Next, consider the multiplication of two n-bit integers a 
and b. The conventional algorithm (used when multiplying with pencil and paper) works as 
follows. Using the distributive law, we see that 


ab= a(by2° + b,2! oe bn—12"—1) 
= a(bo2°) + a(by2!) + «++ + a(by_12"7!). 


We can compute ab using this equation. We first note that abj = a if bj = 1 and ab; = 0 if 

b; = 0. Each time we multiply a term by 2, we shift its binary expansion one place to the left 

and add a zero at the tail end of the expansion. Consequently, we can obtain (ab;)2/ by shifting 

the binary expansion of ab; j places to the left, adding j zero bits at the tail end of this binary 

expansion. Finally, we obtain ab by adding the n integers ab;2/, j=0,1,2,...,.n—1. 
Algorithm 3 displays this procedure for multiplication. 
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110 
x 101 


110 
000 
110 


11110 
FIGURE 2 


Multiplying 
(110)2 and (101)2. 


EXAMPLE 11 


ALGORITHM 3 Multiplication of Integers. 


procedure multiply(a, b: positive integers) 
{the binary expansions of a and b are (ayj—1an—2...a1a0)2 
and (b,—1by-2 ...b1bo)2, respectively} 
for j :=Oton—-1 
if b; = 1 thenc; := a shifted j places 


else c; := 0 
{co, C1, --+;Cn—1 are the partial products} 
p:=0 
for j :=0Oton—-1 

Ppi=pt+cj 


return p {p is the value of ab} 


Example 10 illustrates the use of this algorithm. 
Find the product of a = (110)2 and b = (101)2. 
Solution: First note that 

aby - 2° = (110)2- 1 - 2° = (110)p, 

ab, -2' = (110)2- 0-2! = (0000)2, 
and 

aby - 2? = (110)2- 1-2? = (11000)2. 


To find the product, add (110)2, (0000)2, and (11000)2. Carrying out these additions (us- 
ing Algorithm 2, including initial zero bits when necessary) shows that ab = (1 1110). This 
multiplication is displayed in Figure 2. 


Next, we determine the number of additions of bits and shifts of bits used by Algorithm 3 
to multiply two integers. 


How many additions of bits and shifts of bits are used to multiply a and b using Algorithm 3? 


Solution: Algorithm 3 computes the products of a and b by adding the partial products 


co, C1, €2,..., and cy_1. When b; = 1, we compute the partial product c; by shifting the binary 
expansion of a by j bits. When b; = 0, no shifts are required because c; = 0. Hence, to find 
all n of the integers ab;2/, j =0,1,...,n — 1, requires at most 


O+F1+24+---+n-1 


shifts. Hence, by Example 5 in Section 3.2 the number of shifts required is O(n7). 

To add the integers ab; from j = 0 to 7 =n — 1 requires the addition of an n-bit integer, 
an (n + 1)-bit integer, ..., and a (27)-bit integer. We know from Example 9 that each of these 
additions requires O(n) additions of bits. Consequently, a total of O(n?) additions of bits are 
required for all n additions. < 


Surprisingly, there are more efficient algorithms than the conventional algorithm for mul- 
tiplying integers. One such algorithm, which uses O(n!*>) bit operations to multiply n-bit 
numbers, will be described in Section 8.3. 
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ALGORITHM FOR div AND mod_ Given integers a and d, d > 0, we can find g = 
a div d and r =a mod d using Algorithm 4. In this brute-force algorithm, when a is pos- 
itive we subtract d from a as many times as necessary until what is left is less than d. The 
number of times we perform this subtraction is the quotient and what is left over after all these 
subtractions is the remainder. Algorithm 4 also covers the case where a is negative. This algo- 
rithm finds the quotient g and remainder r when |a| is divided by d. Then, when a < 0 and 
r > 0, it uses these to find the quotient —(g + 1) and remainder d — r when a is divided by d. 
We leave it to the reader (Exercise 59) to show that, assuming that a > d, this algorithm uses 
O(q log a) bit operations. 


ALGORITHM 4 Computing div and mod. 


procedure division algorithm(a: integer, d: positive integer) 


q:=0 

r:=|a| 

while r > d 
ri=r-—d 
q:i=qt+l 

if a < Oandr > 0 then 
ri=d-r 
qg:=-q+)) 


return (q, 7) {q¢ =a div d is the quotient, r = a mod d is the remainder} 


There are more efficient algorithms than Algorithm 4 for determining the quotient g = 
a div d and the remainder r = a mod d when a positive integer a is divided by a positive 
integer d (see [Kn98] for details). These algorithms require O(log a - log d) bit operations. If 
both of the binary expansions of a and d contain n or fewer bits, then we can replace log a - logd 
by n*. This means that we need O(n”) bit operations to find the quotient and remainder when 
a is divided by d. 


Modular Exponentiation 


In cryptography it is important to be able to find b” mod m efficiently, where b, n, and m are 
large integers. It is impractical to first compute b” and then find its remainder when divided 
by m because b” will be a huge number. Instead, we can use an algorithm that employs the 
binary expansion of the exponent n. 

Before we present this algorithm, we illustrate its basic idea. We will explain how to use 
the binary expansion of n, say n = (ag_| ...d,aQ)2, to compute b”. First, note that 


b” = pak-1 28 +-+a1-2+a9 = pave is . pa : pb”. 


This shows that to compute b”, we need only compute the values of b, b?, (b?)? = b+, (b*)? = 


b8,..., b. Once we have these values, we multiply the terms b2’ in this list, where aj = 1. (For 
efficiency, after multiplying by each term, we reduce the result modulo m.) This gives us b”. For 
example, to compute 3!! we first note that 11 = (1011), so that 3!! = 3833!. By successively 
squaring, we find that 3 = 9, 34 = 9 = 81, and 38 = (1)? = 6561. Consequently, 31 = 
38343! = 0561+ 9+3 = 177,147, 
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Be sure to reduce 
modulo m after each 


The algorithm successively finds b mod m, b? mod m, b* mod m,..., b2* ‘mod m 


and multiplies together those terms b”” mod m where a j = 1, finding the remainder 

® of the product when divided by m after each multiplication. Pseudocode for this algorithm 
is shown in Algorithm 5. Note that in Algorithm 5 we can use the most efficient algorithm 
available to compute values of the mod function, not necessarily Algorithm 4. 


multiplication! 


ALGORITHM 5 Modular Exponentiation. 


procedure modular exponentiation(b: integer, n = (dg, dg—2...a\a0)2, 
m: positive integers) 
x1 
power := bmod m 
fori :=Otok—-1 
if a; = | then x := (x- power) mod m 
power := (power - power) mod m 
return x {x equals b” mod m} 


We illustrate how Algorithm 5 works in Example 12. 


EXAMPLE 12 Use Algorithm 5 to find 3° mod 645. 


Solution: Algorithm 5 initially sets x = 1 and power = 3 mod 645 = 3. In the computation 
of 3°4 mod 645, this algorithm determines 3*’ mod 645 for j = 1, 2, ..., 9 by successively 
squaring and reducing modulo 645. If a; = 1 (where a; is the bit in the jth position in the 


binary expansion of 644, which is (1010000100)2), it multiplies the current value of x by 32! 
mod 645 and reduces the result modulo 645. Here are the steps used: 


0 
i=1 
i= 

i=3 
i= 

i=s 
(=o 
i=7 
a 


: Because ap = 0, we have x = | and power = 32 mod 645 = 9 mod 645 = 9; 

: Because a; = 0, we have x = | and power = 9? mod 645 = 81 mod 645 = 81; 

= 2: Because a2 = 1, we have x = | - 81 mod 645 = 81 and power = 812 mod 645 = 6561 mod 645 = 111; 
: Because a3 = 0, we have x = 81 and power = 1117 mod 645 = 12,321 mod 645 = 66; 

= 4: Because a4 = 0, we have x = 81 and power = 66% mod 645 = 4356 mod 645 = 486; 

: Because as = 0, we have x = 81 and power = 4862 mod 645 = 236,196 mod 645 = 126; 

: Because ag = 0, we have x = 81 and power = 126? mod 645 = 15,876 mod 645 = 396; 

: Because a7 = 1, we find that x = (81 - 396) mod 645 = 471 and power = 3967 mod 645 = 156,816 


= 8: Because ag = 0, we have x = 471 and power = 812 mod 645 = 6561 mod 645 = 111: 
= 9: Because do = 1, we find that x = (471 - 111) mod 645 = 36. 


mod 645 = 81; 


This shows that following the steps of Algorithm 5 produces the result 3°+ mod 645 = 36. 
< 


Algorithm 5 is quite efficient; it uses O((log m)* log n) bit operations to find b” mod m (see 
Exercise 58). 


Exercises 


12. 


13. 


14. 


15. 


. Convert the decimal expansion of each of these integers 


to a binary expansion. 


a) 231 b) 4532 c) 97644 


. Convert the decimal expansion of each of these integers 


to a binary expansion. 


a) 321 b) 1023 c) 100632 


. Convert the binary expansion of each of these integers to 


a decimal expansion. 
a) (1 1111)2 
c) (10101 0101). 


b) (10 0000 0001). 
d) (110 1001 0001 0000). 


. Convert the binary expansion of each of these integers to 


a decimal expansion. 
a) (1 1011)2 
c) (11 1011 1110). 


b) (10 1011 0101). 
d) (111 1100 0001 1111)2 


. Convert the octal expansion of each of these integers to a 


binary expansion. 
a) (572)g 
c) (423) 


b) (1604)g 
d) (2417) 


. Convert the binary expansion of each of these integers to 


an octal expansion. 

a) (1111 0111)2 

b) (1010 1010 1010)2 

ce) (111 0111 0111 0111). 
d) (101 0101 0101 0101). 


. Convert the hexadecimal expansion of each of these in- 


tegers to a binary expansion. 
a) (80E)16 
c) (ABBA)i6 


b) (135AB)16 
d) (DEFACED)|6 


. Convert (BADFACED);6 from its hexadecimal expan- 


sion to its binary expansion. 


. Convert (ABCDEF) 16 from its hexadecimal expansion to 


its binary expansion. 


. Convert each of the integers in Exercise 6 from a binary 


expansion to a hexadecimal expansion. 


- Convert (1011 0111 1011) from its binary expansion to 


its hexadecimal expansion. 


Convert (1 1000 0110 0011). from its binary expansion 
to its hexadecimal expansion. 


Show that the hexadecimal expansion of a positive integer 
can be obtained from its binary expansion by grouping to- 
gether blocks of four binary digits, adding initial zeros if 
necessary, and translating each block of four binary digits 
into a single hexadecimal digit. 


Show that the binary expansion of a positive integer can 
be obtained from its hexadecimal expansion by translat- 
ing each hexadecimal digit into a block of four binary 
digits. 

Show that the octal expansion of a positive integer can be 
obtained from its binary expansion by grouping together 
blocks of three binary digits, adding initial zeros if nec- 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 
26. 
27. 
28. 
29. 
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essary, and translating each block of three binary digits 
into a single octal digit. 


Show that the binary expansion of a positive integer can 
be obtained from its octal expansion by translating each 
octal digit into a block of three binary digits. 


Convert (7345321)g to its binary expansion and 
(10 1011 1011)> to its octal expansion. 


Give a procedure for converting from the hexadecimal ex- 
pansion of an integer to its octal expansion using binary 
notation as an intermediate step. 


Give a procedure for converting from the octal expansion 
of an integer to its hexadecimal expansion using binary 
notation as an intermediate step. 


Explain how to convert from binary to base 64 expan- 
sions and from base 64 expansions to binary expansions 
and from octal to base 64 expansions and from base 64 
expansions to octal expansions. 


Find the sum and the product of each of these pairs of 
numbers. Express your answers as a binary expansion. 

a) (100 0111)2, (111 0111)2 

b) (1110 1111)2, 1011 1101). 

c) (10 1010 1010), (1 1111 0000). 

d) (10 0000 0001)s, (11 1111 1111). 


Find the sum and product of each of these pairs of num- 
bers. Express your answers as a base 3 expansion. 

a) (112)3, (210)3 

b) (2112)3, (12021)3 

ce) (20001)3, (1111)3 

d) (120021)3, (2002)3 

Find the sum and product of each of these pairs of num- 
bers. Express your answers as an octal expansion. 

a) (763)g, (147)g 

b) (6001)s, (272) 

c) (1111)s, (777) 

d) (54321), (3456)g 

Find the sum and product of each of these pairs of num- 


bers. Express your answers as a hexadecimal expan- 
sion. 


a) (1AE)16, (BBC)j6 

b) (20CBA)j6, (A01)16 

c) (ABCDE) 16, (111116 

d) (E0000E)\5, (BAAA) 6 

Use Algorithm 5 to find 74 mod 645. 
Use Algorithm 5 to find 11°4 mod 645. 
Use Algorithm 5 to find 3293 mod 99. 
Use Algorithm 5 to find 123!0! mod 101. 


Show that every positive integer can be represented 
uniquely as the sum of distinct powers of 2. [Hint: Con- 
sider binary expansions of integers. ] 
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30. It can be shown that every integer can be uniquely repre- 
sented in the form 


en3* + en 13*-! ++. +.e13 +0, 


where e; = —1,0, or | for j =0,1,2,...,k. Expan- 
sions of this type are called balanced ternary expan- 
sions. Find the balanced ternary expansions of 


a)5. b) 13. c) 37. d) 79. 


31. Show that a positive integer is divisible by 3 if and only 
if the sum of its decimal digits is divisible by 3. 


32. Show that a positive integer is divisible by 11 if and only 
if the difference of the sum of its decimal digits in even- 
numbered positions and the sum of its decimal digits in 
odd-numbered positions is divisible by 11. 


33. Show that a positive integer is divisible by 3 if and only 
if the difference of the sum of its binary digits in even- 
numbered positions and the sum of its binary digits in 
odd-numbered positions is divisible by 3. 

One’s complement representations of integers are used to 
simplify computer arithmetic. To represent positive and nega- 
tive integers with absolute value less than 2”— 1 , atotal of n bits 
is used. The leftmost bit is used to represent the sign. A 0 bit 
in this position is used for positive integers, and a 1 bit in this 
position is used for negative integers. For positive integers, 
the remaining bits are identical to the binary expansion of the 
integer. For negative integers, the remaining bits are obtained 
by first finding the binary expansion of the absolute value of 
the integer, and then taking the complement of each of these 
bits, where the complement of a 1 is a 0 and the complement 
of aQisal. 


34. Find the one’s complement representations, using bit 
strings of length six, of the following integers. 


a) 22 b) 31 c) —7 d) —19 


35. What integer does each of the following one’s comple- 
ment representations of length five represent? 


a) 11001 b) 01101 
c) 10001 d) 11111 


36. If m is a positive integer less than 2”~!, how is the 
one’s complement representation of —m obtained from 
the one’s complement of m, when bit strings of length n 
are used? 


37. How is the one’s complement representation of the sum 
of two integers obtained from the one’s complement rep- 
resentations of these integers? 


38. How is the one’s complement representation of the differ- 
ence of two integers obtained from the one’s complement 
representations of these integers? 


39. Show that the integer m with one’s complement 
representation (d,—1dj—-2...a,a9) can be found us- 
ing the equation m = —ay,_,(2"—! — 1) + Gy_22" 2 + 
+++ a,-2+a. 

Two’s complement representations of integers are also used 

to simplify computer arithmetic and are used more commonly 


than one’s complement representations. To represent an inte- 
ger x with —2"-! <x <2"-!~—1 for a specified positive 
integer n, a total of n bits is used. The leftmost bit is used to 
represent the sign. A 0 bit in this position is used for positive 
integers, and a | bit in this position is used for negative inte- 
gers, just as in one’s complement expansions. For a positive 
integer, the remaining bits are identical to the binary expan- 
sion of the integer. For a negative integer, the remaining bits 
are the bits of the binary expansion of 2”! — |x|. Two’s com- 
plement expansions of integers are often used by computers 
because addition and subtraction of integers can be performed 
easily using these expansions, where these integers can be ei- 
ther positive or negative. 


40. Answer Exercise 34, but this time find the two’s comple- 
ment expansion using bit strings of length six. 


41. Answer Exercise 35 if each expansion is a two’s comple- 
ment expansion of length five. 

42. Answer Exercise 36 for two’s complement expansions. 

43. Answer Exercise 37 for two’s complement expansions. 

44. Answer Exercise 38 for two’s complement expansions. 

45. Show that the integer m with two’s complement 
representation (d,—1d,—-2...a,a9) can be found us- 
ing the equationm = —a,_, -2"~!+ Gn—22" 2 feet 
a,-2+ ao. 

46. Give a simple algorithm for forming the two’s comple- 
ment representation of an integer from its one’s comple- 
ment representation. 

47. Sometimes integers are encoded by using four-digit bi- 
nary expansions to represent each decimal digit. This pro- 
duces the binary coded decimal form of the integer. For 
instance, 791 is encoded in this way by 011110010001. 
How many bits are required to represent a number with 
n decimal digits using this type of encoding? 

A Cantor expansion is a sum of the form 


ann! + dn-1(n — I)! +--+ +a22! + az1!, 


where a; is an integer with O < aj <i fori = 1, 2,...,n. 
48. Find the Cantor expansions of 
a) 2. b) 7. 
ce) 19. d) 87. 
e) 1000. f) 1,000,000. 
* 49, Describe an algorithm that finds the Cantor expansion of 
an integer. 


*50. Describe an algorithm to add two integers from their Can- 

tor expansions. 

51. Add (10111)2 and (11010). by working through each 
step of the algorithm for addition given in the text. 

52. Multiply (1110)2 and (1010). by working through each 
step of the algorithm for multiplication given in the text. 

53. Describe an algorithm for finding the difference of two 
binary expansions. 


54. Estimate the number of bit operations used to subtract 
two binary expansions. 
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55. Devise an algorithm that, given the binary expansions of 57. Estimate the complexity of Algorithm 1 for finding the 
the integers a and b, determines whether a > b, a = b, base b expansion of an integer n in terms of the number 
ora <b. of divisions used. 

*58. Show that Algorithm 5 uses O ((log m)* log n) bit opera- 

56. How many bit operations does the comparison algo- tions to find b” mod m. 
rithm from Exercise 55 use when the larger of a and b 59. Show that Algorithm 4 uses O(q loga) bit operations, 
has n bits in its binary expansion? assuming that a > d. 


Primes and Greatest Common Divisors 


DEFINITION 1 


EXAMPLE 1 


Introduction 


In Section 4.1 we studied the concept of divisibility of integers. One important concept based 
on divisibility is that of a prime number. A prime is an integer greater than | that is divisible by 
no positive integers other than | and itself. The study of prime numbers goes back to ancient 
times. Thousands of years ago it was known that there are infinitely many primes; the proof of 
this fact, found in the works of Euclid, is famous for its elegance and beauty. 

We will discuss the distribution of primes among the integers. We will describe some 
of the results about primes found by mathematicians in the last 400 years. In particular, we 
will introduce an important theorem, the fundamental theorem of arithmetic. This theorem, 
which asserts that every positive integer can be written uniquely as the product of primes in 
nondecreasing order, has many interesting consequences. We will also discuss some of the many 
old conjectures about primes that remain unsettled today. 

Primes have become essential in modern cryptographic systems, and we will develop some 
of their properties important in cryptography. For example, finding large primes is essential in 
modern cryptography. The length of time required to factor large integers into their prime factors 
is the basis for the strength of some important modern cryptographic systems. 

In this section we will also study the greatest common divisor of two integers, as well as the 
least common multiple of two integers. We will develop an important algorithm for computing 
greatest common divisors, called the Euclidean algorithm. 


Primes 
Every integer greater than | is divisible by at least two integers, because a positive integer is 


divisible by | and by itself. Positive integers that have exactly two different positive integer 
factors are called primes. 


An integer p greater than | is called prime if the only positive factors of p are 1 and p. 
A positive integer that is greater than | and is not prime is called composite. 


Remark: The integer n is composite if and only if there exists an integer a such that a | n and 
l<a<n. 


The integer 7 is prime because its only positive factors are 1 and 7, whereas the integer 9 is 
composite because it is divisible by 3. < 


The primes are the building blocks of positive integers, as the fundamental theorem of 
arithmetic shows. The proof will be given in Section 5.2. 
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THEOREM 2 


EXAMPLE 3 


THE FUNDAMENTAL THEOREM OF ARITHMETIC Every integer greater than 1 
can be written uniquely as a prime or as the product of two or more primes where the prime 
factors are written in order of nondecreasing size. 


Example 2 gives some prime factorizations of integers. 
The prime factorizations of 100, 641, 999, and 1024 are given by 


100 =2-2-5-5=275?, 

641 = 641, 

999 = 3-3-3.37 =3? - 37, 

W024 = 95.929 295949292319. 9 90, 


Trial Division 


It is often important to show that a given integer is prime. For instance, in cryptology, large 
primes are used in some methods for making messages secret. One procedure for showing that 
an integer is prime is based on the following observation. 


If n is a composite integer, then n has a prime divisor less than or equal to ./n. 


Proof: If n is composite, by the definition of a composite integer, we know that it has a factor 
a with | <a <n. Hence, by the definition of a factor of a positive integer, we have n = ab, 
where b is a positive integer greater than 1. We will show thata < /norb < /n.Ifa > ./nand 
b> Jn, thenab > ./n-./n =n, whichis a contradiction. Consequently, a < ./n or b < ./n. 
Because both a and b are divisors of n, we see that n has a positive divisor not exceeding \/n. 
This divisor is either prime or, by the fundamental theorem of arithmetic, has a prime divisor 
less than itself. In either case, n has a prime divisor less than or equal to ./n. | 


From Theorem 2, it follows that an integer is prime if it is not divisible by any prime less 
than or equal to its square root. This leads to the brute-force algorithm known as trial division. 
To use trial division we divide n by all primes not exceeding ./n and conclude that n is prime 
if it is not divisible by any of these primes. In Example 3 we use trial division to show that 101 
is prime. 


Show that 101 is prime. 


Solution: The only primes not exceeding /101 are 2, 3, 5, and 7. Because 101 is not divisible 
by 2, 3, 5, or 7 (the quotient of 101 and each of these integers is not an integer), it follows that 
101 is prime. < 


Because every integer has a prime factorization, it would be useful to have a procedure for 
finding this prime factorization. Consider the problem of finding the prime factorization of n. 
Begin by dividing n by successive primes, starting with the smallest prime, 2. If n has a prime 
factor, then by Theorem 3 a prime factor p not exceeding ./n will be found. So, if no prime 
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factor not exceeding ./n is found, then n is prime. Otherwise, if a prime factor p is found, 
continue by factoring n/p. Note that n/p has no prime factors less than p. Again, if n/p has 
no prime factor greater than or equal to p and not exceeding its square root, then it is prime. 
Otherwise, if it has a prime factor g, continue by factoring n/(pq). This procedure is continued 
until the factorization has been reduced to a prime. This procedure is illustrated in Example 4. 


Find the prime factorization of 7007. 


Solution: To find the prime factorization of 7007, first perform divisions of 7007 by succes- 
sive primes, beginning with 2. None of the primes 2, 3, and 5 divides 7007. However, 7 di- 
vides 7007, with 7007/7 = 1001. Next, divide 1001 by successive primes, beginning with 7. 
It is immediately seen that 7 also divides 1001, because 1001/7 = 143. Continue by divid- 
ing 143 by successive primes, beginning with 7. Although 7 does not divide 143, 11 does 
divide 143, and 143/11 = 13. Because 13 is prime, the procedure is completed. It follows that 
7007 = 7- 1001 = 7-7-143 =7-7-11- 13. Consequently, the prime factorization of 7007 
i677 ols 7 «1143, < 


Prime numbers were studied in ancient times for philosophical reasons. Today, there are 
highly practical reasons for their study. In particular, large primes play a crucial role in cryp- 
tography, as we will see in Section 4.6. 


The Sieve of Eratosthenes 


Note that composite integers not exceeding 100 must have a prime factor not exceeding 10. 
Because the only primes less than 10 are 2, 3, 5, and 7, the primes not exceeding 100 are these 
four primes and those positive integers greater than | and not exceeding 100 that are divisible 
by none of 2, 3,5, or 7. 


The sieve of Eratosthenes is used to find all primes not exceeding a specified positive 
integer. For instance, the following procedure is used to find the primes not exceeding 100. We 
begin with the list of all integers between | and 100. To begin the sieving process, the integers 
that are divisible by 2, other than 2, are deleted. Because 3 is the first integer greater than 2 that 
is left, all those integers divisible by 3, other than 3, are deleted. Because 5 is the next integer 
left after 3, those integers divisible by 5, other than 5, are deleted. The next integer left is 7, 
so those integers divisible by 7, other than 7, are deleted. Because all composite integers not 
exceeding 100 are divisible by 2, 3, 5, or 7, all remaining integers except | are prime. In Table 1, 
the panels display those integers deleted at each stage, where each integer divisible by 2, other 
than 2, is underlined in the first panel, each integer divisible by 3, other than 3, is underlined 
in the second panel, each integer divisible by 5, other than 5, is underlined in the third panel, 
and each integer divisible by 7, other than 7, is underlined in the fourth panel. The integers not 
underlined are the primes not exceeding 100. We conclude that the primes less than 100 are 2, 
3,5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, and 97. 


THE INFINITUDE OF PRIMES | Ithas long been known that there are infinitely many primes. 
This means that whenever p1, p2,..., Pn are the n smallest primes, we know there is a larger 


ERATOSTHENES (276 B.c.£.-194 B.c.£.) It is known that Eratosthenes was born in Cyrene, a Greek colony 
west of Egypt, and spent time studying at Plato’s Academy in Athens. We also know that King Ptolemy II 
invited Eratosthenes to Alexandria to tutor his son and that later Eratosthenes became chief librarian at the 
famous library at Alexandria, a central repository of ancient wisdom. Eratosthenes was an extremely versatile 
scholar, writing on mathematics, geography, astronomy, history, philosophy, and literary criticism. Besides his 
work in mathematics, he is most noted for his chronology of ancient history and for his famous measurement 
of the size of the earth. 
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TABLE 1 The Sieve of Eratosthenes. 


Integers divisible by 2 other than 2 Integers divisible by 3 other than 3 


receive an underline. receive an underline. 


Integers divisible by 5 other than 5 Integers divisible by 7 other than 7 receive 


receive an underline. an underline; integers in color are prime. 
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prime not listed. We will prove this fact using a proof given by Euclid in his famous mathematics 
text, The Elements. This simple, yet elegant, proof is considered by many mathematicians to be 
among the most beautiful proofs in mathematics. It is the first proof presented in the book Proofs 
from THE BOOK[AiZi10], where THE BOOK refers to the imagined collection of perfect proofs 
that the famous mathematician Paul Erdés claimed is maintained by God. By the way, there 
are a vast number of different proofs than there are an infinitude of primes, and new ones are 
published surprisingly frequently. 


There are infinitely many primes. 


Proof: We will prove this theorem using a proof by contradiction. We assume that there are only 
finitely many primes, p1, p2,.-.., Pn- Let 


O= pip2-:-Pn +l. 


By the fundamental theorem of arithmetic, Q is prime or else it can be written as the product of 
two or more primes. However, none of the primes p; divides Q, for if p; | Q, then p; divides 
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Q — pip2--: Pn = 1. Hence, there is a prime not in the list pj, p2,..., Py. This prime is 
either Q, if it is prime, or a prime factor of Q. This is a contradiction because we assumed that 
we have listed all the primes. Consequently, there are infinitely many primes. | 


Remark: Note that in this proof we do not state that Q is prime! Furthermore, in this proof, we 
have given a nonconstructive existence proof that given any n primes, there is a prime not in 
this list. For this proof to be constructive, we would have had to explicitly give a prime not in 
our original list of n primes. 


Because there are infinitely many primes, given any positive integer there are primes greater 
than this integer. There is an ongoing quest to discover larger and larger prime numbers; for 
almost all the last 300 years, the largest prime known has been an integer of the special form 
2? — 1, where p is also prime. (Note that 2” — 1 cannot be prime when n is not prime; see 
Exercise 9.) Such primes are called Mersenne primes, after the French monk Marin Mersenne, 
who studied them in the seventeenth century. The reason that the largest known prime has usually 
been a Mersenne prime is that there is an extremely efficient test, known as the Lucas—Lehmer 
test, for determining whether 2? — 1 is prime. Furthermore, it is not currently possible to test 
numbers not of this or certain other special forms anywhere near as quickly to determine whether 
they are prime. 


EXAMPLE 5. The numbers 2? — 1 = 3, 2? — 1 =7, 2° — 1 = 31 and 2’ — 1 = 127 are Mersenne primes, 
while 2!! — 1 = 2047 is not a Mersenne prime because 2047 = 23 - 89. < 


Progress in finding Mersenne primes has been steady since computers were invented. As of 
early 2011, 47 Mersenne primes were known, with 16 found since 1990. The largest Mersenne 
Links prime known (again as of early 2011) is 2*3-!!7:6°? — 1, a number with nearly 13 million decimal 
digits, which was shown to be prime in 2008. A communal effort, the Great Internet Mersenne 
Prime Search (GIMPS), is devoted to the search for new Mersenne primes. You can join this 
search, and if you are lucky, find a new Mersenne prime and possibly even win a cash prize. By 
the way, even the search for Mersenne primes has practical implications. One quality control test 
for supercomputers has been to replicate the Lucas—Lehmer test that establishes the primality of 
a large Mersenne prime. (See [Ro10] for more information about the quest for finding Mersenne 
primes.) 


THE DISTRIBUTION OF PRIMES Theorem 3 tells us that there are infinitely many primes. 
However, how many primes are less than a positive number x? This question interested mathe- 
maticians for many years; in the late eighteenth century, mathematicians produced large tables 


Links 


MARIN MERSENNE (1588-1648) Mersenne was born in Maine, France, into a family of laborers and 
attended the College of Mans and the Jesuit College at La Fléche. He continued his education at the Sor- 
bonne, studying theology from 1609 to 1611. He joined the religious order of the Minims in 1611, a group 
whose name comes from the word minimi (the members of this group were extremely humble; they consid- 
ered themselves the least of all religious orders). Besides prayer, the members of this group devoted their 
| energy to scholarship and study. In 1612 he became a priest at the Place Royale in Paris; between 1614 and 
1618 he taught philosophy at the Minim Convent at Nevers. He returned to Paris in 1619, where his cell 
in the Minims de I’Annociade became a place for meetings of French scientists, philosophers, and mathe- 
maticians, including Fermat and Pascal. Mersenne corresponded extensively with scholars throughout Europe, 
serving as a clearinghouse for mathematical and scientific knowledge, a function later served by mathematical journals (and today 
also by the Internet). Mersenne wrote books covering mechanics, mathematical physics, mathematics, music, and acoustics. He 
studied prime numbers and tried unsuccessfully to construct a formula representing all primes. In 1644 Mersenne claimed that 
2? — 1 is prime for p = 2, 3, 5, 7, 13, 17, 19, 31, 67, 127, 257 but is composite for all other primes less than 257. It took over 300 
years to determine that Mersenne’s claim was wrong five times. Specifically, 2? — 1 is not prime for p = 67 and p = 257 but is 
prime for p = 61, p = 87, and p = 107. It is also noteworthy that Mersenne defended two of the most famous men of his time, 
Descartes and Galileo, from religious critics. He also helped expose alchemists and astrologers as frauds. 
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of prime numbers to gather evidence concerning the distribution of primes. Using this evidence, 
the great mathematicians of the day, including Gauss and Legendre, conjectured, but did not 
prove, Theorem 4. 


THE PRIME NUMBER THEOREM The ratio of the number of primes not exceeding 
x and x/In x approaches | as x grows without bound. (Here In x is the natural logarithm 
of x.) 


The prime number theorem was first proved in 1896 by the French mathematician Jacques 
Hadamard and the Belgian mathematician Charles-Jean-Gustave-Nicholas de la Vallée-Poussin 
using the theory of complex variables. Although proofs not using complex variables have been 
found, all known proofs of the prime number theorem are quite complicated. 

We can use the prime number theorem to estimate the odds that a randomly chosen number 
is prime. The prime number theorem tells us that the number of primes not exceeding x can be 
approximated by x /In x. Consequently, the odds that a randomly selected positive integer less 
than n is prime are approximately (n/Inn)/n = 1/Inn. Sometimes we need to find a prime 
with a particular number of digits. We would like an estimate of how many integers with a 
particular number of digits we need to select before we encounter a prime. Using the prime 
number theorem and calculus, it can be shown that the probability that an integer n is prime 
is also approximately 1/In n. For example, the odds that an integer near 10!° is prime are 
approximately 1/1In 10!°°, which is approximately 1/2300. (Of course, by choosing only odd 
numbers, we double our chances of finding a prime.) 

Using trial division with Theorem 2 gives procedures for factoring and for primality testing. 
However, these procedures are not efficient algorithms; many much more practical and efficient 
algorithms for these tasks have been developed. Factoring and primality testing have become 
important in the applications of number theory to cryptography. This has led to a great interest 
in developing efficient algorithms for both tasks. Clever procedures have been devised in the 
last 30 years for efficiently generating large primes. Moreover, in 2002, an important theoretical 
discovery was made by Manindra Agrawal, Neeraj Kayal, and Nitin Saxena. They showed there 
is a polynomial-time algorithm in the number of bits in the binary expansion of an integer for 
determining whether a positive integer is prime. Algorithms based on their work use O ((logn)°) 
bit operations to determine whether a positive integer n is prime. 

However, even though powerful new factorization methods have been developed in the 
same time frame, factoring large numbers remains extraordinarily more time-consuming than 
primality testing. No polynomial-time algorithm for factoring integers is known. Nevertheless, 
the challenge of factoring large numbers interests many people. There is a communal effort on 
the Internet to factor large numbers, especially those of the special form k” + 1, where k is a 
small positive integer and n is a large positive integer (such numbers are called Cunningham 
numbers). At any given time, there is a list of the “Ten Most Wanted” large numbers of this type 
awaiting factorization. 


PRIMES AND ARITHMETIC PROGRESSIONS Every odd integer is in one of the two 
arithmetic progressions 4k + lor4k + 3,k = 1,2, .... Because we know that there are infinitely 
many primes, we can ask whether there are infinitely many primes in both of these arithmetic 
progressions. The primes 5, 13, 17, 29, 37, 41,... are in the arithmetic progression 4k + 1; 
the primes 3, 7, 11, 19, 23, 31, 43,... are in the arithmetic progression 4k + 3. Looking at 
the evidence hints that there may be infinitely many primes in both progressions. What about 
other arithmetic progressions ak + b, k = 1,2,..., where no integer greater than one divides 
both a and b? Do they contain infinitely many primes? The answer was provided by the German 
mathematician G. Lejeune Dirichlet, who proved that every such arithmetic progression contains 
infinitely many primes. His proof, and all proofs found later, are beyond the scope of this book. 
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However, it is possible to prove special cases of Dirichlet’s theorem using the ideas developed 
in this book. For example, Exercises 54 and 55 ask for proofs that there are infinitely many 
primes in the arithmetic progressions 3k + 2 and 4k + 3, where k is a positive integer. (The hint 
for each of these exercises supplies the basic idea needed for the proof.) 

We have explained that every arithmetic progression ak + b, k = 1,2,..., where a and b 
have no common factor greater than one, contains infinitely many primes. But are there long 
arithmetic progressions made up of just primes? For example, some exploration shows that 5, 
11, 17, 23, 29 is an arithmetic progression of five primes and 199, 409, 619, 829, 1039, 1249, 
1459, 1669, 1879, 2089 is an arithmetic progression of ten primes. In the 1930s, the famous 
mathematician Paul Erdés conjectured that for every positive integer n greater than two, there 
is an arithmetic progression of length n made up entirely of primes. In 2006, Ben Green and 
Terence Tao were able to prove this conjecture. Their proof, considered to be a mathematical 
tour de force, is a nonconstructive proof that combines powerful ideas from several advanced 
areas of mathematics. 


Conjectures and Open Problems About Primes 


Number theory is noted as a subject for which it is easy to formulate conjectures, some of which 
are difficult to prove and others that remained open problems for many years. We will describe 
some conjectures in number theory and discuss their status in Examples 6-9. 


It would be useful to have a function f (n) such that f (7) is prime for all positive integers n. If we 
had such a function, we could find large primes for use in cryptography and other applications. 
Looking for such a function, we might check out different polynomial functions, as some 
mathematicians did several hundred years ago. After a lot of computation we may encounter 
the polynomial f(n) = n? — n +41. This polynomial has the interesting property that f(n) is 
prime for all positive integers n not exceeding 40. [We have f(1) = 41, f(2) = 43, f(3) = 47, 
Ff (4) = 53, and so on.] This can lead us to the conjecture that f(n) is prime for all positive 
integers n. Can we settle this conjecture? 


Solution: Perhaps not surprisingly, this conjecture turns out to be false; we do not have to look far 
to find a positive integer n for which f (n) is composite, because f (41) = 417 — 41+ 41 = 417. 
Because f(n) =n? —n-+ 41 is prime for all positive integers n with 1 <n < 40, we might 


TERENCE TAO (BORN 1975) Tao was born in Australia. His father is a pediatrician and his mother taught 
mathematics at a Hong Kong secondary school. Tao was a child prodigy, teaching himself arithmetic at the age 
of two. At 10, he became the youngest contestant at the International Mathematical Olympiad (IMO); he won 
an IMO gold medal at 13. Tao received his bachelors and masters degrees when he was 17, and began graduate 
studies at Princeton, receiving his Ph.D. in three years. In 1996 he became a faculty member at UCLA, where 
he continues to work. 

Tao is extremely versatile; he enjoys working on problems in diverse areas, including harmonic analy- 
sis, partial differential equations, number theory, and combinatorics. You can follow his work by reading his 
blog where he discusses progress on various problems. His most famous result is the Green-Tao theorem, 


which says that there are arbitrarily long arithmetic progressions of primes. Tao has made important contributions to the applications 
of mathematics, such as developing a method for reconstructing digital images using the least possible amount of information. 

Tao has an amazing reputation among mathematicians; he has become a Mr. Fix-It for researchers in mathematics. The well-known 
mathematician Charles Fefferman, himself a child prodigy, has said that “if you’re stuck on a problem, then one way out is to interest 
Terence Tao.” In 2006 Tao was awarded a Fields Medal, the most prestigious award for mathematicians under the age of 40. He 
was also awarded a MacArthur Fellowship in 2006, and in 2008, he received the Allan T. Waterman award, which came with a 
$500,000 cash prize to support research work of scientists early in their career. Tao’s wife Laura is an engineer at the Jet Propulsion 


Laboratory. 
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be tempted to find a different polynomial with the property that f (7) is prime for all positive 
integers n. However, there is no such polynomial. It can be shown that for every polynomial 
jf) with integer coefficients, there is a positive integer y such that f(y) is composite. (See 
Exercise 23 in the Supplementary Exercises.) 


Many famous problems about primes still await ultimate resolution by clever people. We 
describe a few of the most accessible and better known of these open problems in Examples 7-9. 
Number theory is noted for its wealth of easy-to-understand conjectures that resist attack by all 
but the most sophisticated techniques, or simply resist all attacks. We present these conjectures 
to show that many questions that seem relatively simple remain unsettled even in the twenty-first 
century. 


Goldbach’s Conjecture In 1742, Christian Goldbach, in a letter to Leonhard Euler, conjec- 
tured that every odd integer n,n > 5, is the sum of three primes. Euler replied that this conjecture 
is equivalent to the conjecture that every even integer n, n > 2, is the sum of two primes (see 
Exercise 21 in the Supplementary Exercises). The conjecture that every even integern,n > 2, is 
the sum of two primes is now called Goldbach’s conjecture. We can check this conjecture for 
small even numbers. For example, 4 = 2+ 2,6=3+3,8=5+3,10=7+3,12=7+5, 
and so on. Goldbach’s conjecture was verified by hand calculations for numbers up to the mil- 
lions prior to the advent of computers. With computers it can be checked for extremely large 
BED ED As of mid 2011, the conjecture has been checked for all positive even integers up to 
1.6- 10°. 

Although no proof of Goldbach’s conjecture has been found, most mathematicians believe 
it is true. Several theorems have been proved, using complicated methods from analytic number 
theory far beyond the scope of this book, establishing results weaker than Goldbach’s conjecture. 
Among these are the result that every even integer greater than 2 is the sum of at most six primes 
(proved in 1995 by O. Ramaré) and that every sufficiently large positive integer is the sum of a 
prime and a number that is either prime or the product of two primes (proved in 1966 by J. R. 
Chen). Perhaps Goldbach’s conjecture will be settled in the not too distant future. < 


There are many conjectures asserting that there are infinitely many primes of certain special 
forms. A conjecture of this sort is the conjecture that there are infinitely many primes of the form 
n2 + 1, where n is a positive integer. For example, 5 = 9 4: 1li7v= A> 137= G+ 1, and 
so on. The best result currently known is that there are infinitely many positive integers n such 
that n? + 1 is prime or the product of at most two primes (proved by Henryk Iwaniec in 1973 
using advanced techniques from analytic number theory, far beyond the scope of this book). 4 


The Twin Prime Conjecture Twin primes are pairs of primes that differ by 2, such as 3 and 
5,5 and 7, 11 and 13, 17 and 19, and 4967 and 4969. The twin prime conjecture asserts that 
there are infinitely many twin primes. The strongest result proved concerning twin primes is 
that there are infinitely many pairs p and p+ 2, where p is prime and p + 2 is prime or the 
product of two primes (proved by J. R. Chen in 1966). The world’s record for twin primes, as of 
mid 2011, consists of the numbers 65,516,468, 355 - 2793-333 + 1, which have 100,355 decimal 
digits. < 


CHRISTIAN GOLDBACH (1690-1764) Christian Goldbach was born in Konigsberg, Prussia, the city noted for its famous bridge 
problem (which will be studied in Section 10.5). He became professor of mathematics at the Academy in St. Petersburg in 1725. In 
1728 Goldbach went to Moscow to tutor the son of the Tsar. He entered the world of politics when, in 1742, he became a staff member 
in the Russian Ministry of Foreign Affairs. Goldbach is best known for his correspondence with eminent mathematicians, including 
Euler and Bernoulli, for his famous conjectures in number theory, and for several contributions to analysis. 
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Greatest Common Divisors and Least Common Multiples 


The largest integer that divides both of two integers is called the greatest common divisor of 
these integers. 


Let a and b be integers, not both zero. The largest integer d such that d | a andd | bis called 
the greatest common divisor of a and b. The greatest common divisor of a and b is denoted 
by gcd(a, D). 


The greatest common divisor of two integers, not both zero, exists because the set of common 
divisors of these integers is nonempty and finite. One way to find the greatest common divisor 
of two integers is to find all the positive common divisors of both integers and then take the 
largest divisor. This is done in Examples 10 and 11. Later, a more efficient method of finding 
greatest common divisors will be given. 


What is the greatest common divisor of 24 and 36? 
Solution: The positive common divisors of 24 and 36 are 1, 2, 3, 4, 6, and 12. Hence, 
gcd(24, 36) = 12. < 
What is the greatest common divisor of 17 and 22? 


Solution: The integers 17 and 22 have no positive common divisors other than 1, so that 
gcd(17, 22) = 1. < 


Because it is often important to specify that two integers have no common positive divisor 
other than 1, we have Definition 3. 


The integers a and b are relatively prime if their greatest common divisor is 1. 


By Example 11 it follows that the integers 17 and 22 are relatively prime, because 
gced(17, 22) = 1. 


Because we often need to specify that no two integers in a set of integers have a common 
positive divisor greater than 1, we make Definition 4. 


The integers a1, a2,...,n are pairwise relatively prime if gcd(a;,a;) = 1 whenever | < 
i <i Si, 


Determine whether the integers 10, 17, and 21 are pairwise relatively prime and whether the 
integers 10, 19, and 24 are pairwise relatively prime. 


Solution: Because gcd(10, 17) = 1, gcd(10, 21) = 1, and gced(17, 21) = 1, we conclude that 
10, 17, and 21 are pairwise relatively prime. 

Because gcd(10, 24) = 2 > 1, we see that 10, 19, and 24 are not pairwise relatively 
prime. 
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EXAMPLE 14 


DEFINITION 5 


EXAMPLE 15 


Another way to find the greatest common divisor of two positive integers is to use the prime 
factorizations of these integers. Suppose that the prime factorizations of the positive integers a 
and b are 


— plll 742 a — bi bo b 
a= Py Py +++ Py", D= Py Py +++ Ph’s 


where each exponent is a nonnegative integer, and where all primes occurring in the prime 
factorization of either a or b are included in both factorizations, with zero exponents if necessary. 
Then ged(a, b) is given by 


min(a,, b}) . min(az, bz) nes 


gcd(a, b) = p, Po 


min(an, bn) 
n ’ 


P. 


where min(x, y) represents the minimum of the two numbers x and y. To show that this formula 
for gcd(a, b) is valid, we must show that the integer on the right-hand side divides both a and b, 
and that no larger integer also does. This integer does divide both a and b, because the power of 
each prime in the factorization does not exceed the power of this prime in either the factorization 
of a or that of b. Further, no larger integer can divide both a and b, because the exponents of 
the primes in this factorization cannot be increased, and no other primes can be included. 


Because the prime factorizations of 120 and 500 are 120 = 23.3.5 and 500 = 22. 53, the 
greatest common divisor is 


gcd (120, 500) = gminG3, 2)gmin(1, 0) 5min(1, 3) — 97395! = 20. dl 


Prime factorizations can also be used to find the least common multiple of two integers. 


The least common multiple of the positive integers a and b is the smallest positive integer that 
is divisible by both a and b. The least common multiple of a and b is denoted by Icm(a, b). 


The least common multiple exists because the set of integers divisible by both a and b is 
nonempty (as ab belongs to this set, for instance), and every nonempty set of positive integers 
has a least element (by the well-ordering property, which will be discussed in Section 5.2). 
Suppose that the prime factorizations of a and b are as before. Then the least common multiple 
of a and b is given by 


max(a 1, bj) _ max(az, b2) ; 


Icm(a, b) = p, pe ap 


max(dn, by) 
n , 


where max(x, y) denotes the maximum of the two numbers x and y. This formula is valid because 
a common multiple of a and b has at least max(q;, b;) factors of p; in its prime factorization, 
and the least common multiple has no other prime factors besides those in a and b. 


What is the least common multiple of 233°77 and 2433? 
Solution: We have 


Iem(23357?, 273°) = gimax(3, 4) 3max(5, 3)7max(2, 0) = 943572. 4 


Theorem 5 gives the relationship between the greatest common divisor and least common 
multiple of two integers. It can be proved using the formulae we have derived for these quantities. 
The proof of this theorem is left as Exercise 31. 


THEOREM 5 


Links 


4.3 Primes and Greatest Common Divisors 267 


Let a and b be positive integers. Then 


ab = gcd(a, b) - lcm(a, b). 


The Euclidean Algorithm 


Computing the greatest common divisor of two integers directly from the prime factorizations 
of these integers is inefficient. The reason is that it is time-consuming to find prime factoriza- 
tions. We will give a more efficient method of finding the greatest common divisor, called the 
Euclidean algorithm. This algorithm has been known since ancient times. It is named after the 
ancient Greek mathematician Euclid, who included a description of this algorithm in his book 
The Elements. 

Before describing the Euclidean algorithm, we will show how it is used to find ged(91, 287). 
First, divide 287, the larger of the two integers, by 91, the smaller, to obtain 


287 = 91-34 14. 


Any divisor of 91 and 287 must also be a divisor of 287 — 91 . 3 = 14. Also, any divisor of 91 
and 14 must also be a divisor of 287 = 91 . 3 + 14. Hence, the greatest common divisor of 91 
and 287 is the same as the greatest common divisor of 91 and 14. This means that the problem 
of finding gcd(91, 287) has been reduced to the problem of finding gcd(91, 14). 

Next, divide 91 by 14 to obtain 


91 = 14-6+7. 


Because any common divisor of 91 and 14 also divides 91 — 14 - 6 = 7 and any common divisor 
of 14 and 7 divides 91, it follows that gcd(91, 14) = gced(14, 7). 
Continue by dividing 14 by 7, to obtain 


14=7.-2. 


Because 7 divides 14, it follows that gcd(14, 7) = 7. Furthermore, because gcd(287, 91) = 
gcd(91, 14) = gcd(14, 7) = 7, the original problem has been solved. 

We now describe how the Euclidean algorithm works in generality. We will use successive 
divisions to reduce the problem of finding the greatest common divisor of two positive integers 
to the same problem with smaller integers, until one of the integers is zero. 

The Euclidean algorithm is based on the following result about greatest common divisors 
and the division algorithm. 


EUCLID (325 B.c.£.—265 B.c.£.) Euclid was the author of the most successful mathematics book ever written, 
The Elements, which appeared in over 1000 different editions from ancient to modern times. Little is known 
about Euclid’s life, other than that he taught at the famous academy at Alexandria in Egypt. Apparently, Euclid 
did not stress applications. When a student asked what he would get by learning geometry, Euclid explained 
that knowledge was worth acquiring for its own sake and told his servant to give the student a coin “because he 
must make a profit from what he learns.” 
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LEMMA 1 


EXAMPLE 16 


Let a = bq +r, where a, b, q, andr are integers. Then gcd(a, b) = gcd(b, r). 


Proof: If we can show that the common divisors of a and b are the same as the common divisors 
of b and r, we will have shown that gcd(a, b) = gcd(b, r), because both pairs must have the 
same greatest common divisor. 

So suppose that d divides both a and b. Then it follows that d also divides a — bq = r (from 
Theorem | of Section 4.1). Hence, any common divisor of a and D is also a common divisor 
of b andr. 

Likewise, suppose that d divides both b and r. Then d also divides bg + r = a. Hence, any 
common divisor of b and r is also a common divisor of a and b. 

Consequently, gcd(a, b) = gcd(b, r). se] 


Suppose that a and b are positive integers with a > b. Let ro = a and rj = b. When we 
successively apply the division algorithm, we obtain 


mM =nqi tre 0<m <n", 
YY =rqtrs3 0<73 <1, 


ln—2 =!n-19n-1 + Tn 0<ry <lVn-1; 


Tn—-1 =n n- 


Eventually a remainder of zero occurs in this sequence of successive divisions, because the 
sequence of remainders a = r9 > rj > r2 > --- > O cannot contain more than a terms. Fur- 
thermore, it follows from Lemma | that 


gcd(a, b) = ged(ro, r1) = ged(r1, 12) = --- = gcd(mn_2, Mn—1) 
= gcd(rn—1, In) = gcd(rn, 0) =Tn. 


Hence, the greatest common divisor is the last nonzero remainder in the sequence of divisions. 
Find the greatest common divisor of 414 and 662 using the Euclidean algorithm. 
Solution: Successive uses of the division algorithm give: 


662 = 414-1+ 248 
414 = 248-1+ 166 
248 = 166-1+4 82 


166 = 82.242 
82 =2.-41. 
Hence, gcd(414, 662) = 2, because 2 is the last nonzero remainder. < 


The Euclidean algorithm is expressed in pseudocode in Algorithm 1. 
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ALGORITHM 1 The Euclidean Algorithm. 


procedure gcd(a, b: positive integers) 


xi=a 

yi=b 

while y 4 0 
r:=x mod y 
Ly 
yi=r 


return x{gcd(a, b) is x} 


In Algorithm 1, the initial values of x and y are a and J, respectively. At each stage of the 
procedure, x is replaced by y, and y is replaced by x mod y, which is the remainder when «x is 
divided by y. This process is repeated as long as y £ 0. The algorithm terminates when y = 0, 
and the value of x at that point, the last nonzero remainder in the procedure, is the greatest 
common divisor of a and b. 

We will study the time complexity of the Euclidean algorithm in Section 5.3, where we 
will show that the number of divisions required to find the greatest common divisor of a and b, 
where a > b, is O(log D). 


gcds as Linear Combinations 


An important result we will use throughout the remainder of this section is that the greatest 
common divisor of two integers a and b can be expressed in the form 


sa+tb, 


where s and ¢ are integers. In other words, gcd(a, b) can be expressed as a linear combination 
with integer coefficients of a and b. For example, gcd(6, 14) = 2, and 2 = (—2)-6+4+ 1- 14. 
We state this fact as Theorem 6. 


BEZOUT’S THEOREM Ifa and bare positive integers, then there exist integers s and ¢ 
such that gcd(a, b) = sa+ tb. 


ETIENNE BEZOUT (1730-1783) Bézout was born in Nemours, France, where his father was a magistrate. 
Reading the writings of the great mathematician Leonhard Euler enticed him to become a mathematician. In 
1758 he was appointed to a position at the Académie des Sciences in Paris; in 1763 he was appointed examiner 
of the Gardes de la Marine, where he was assigned the task of writing mathematics textbooks. This assignment 
led to a four-volume textbook completed in 1767. Bézout is well known for his six-volume comprehensive 
textbook on mathematics. His textbooks were extremely popular and were studied by many generations of 
students hoping to enter the Ecole Polytechnique, the famous engineering and science school. His books were 
translated into English and used in North America, including at Harvard. 

His most important original work was published in 1779 in the book Théorie générale des équations 


algébriques, where he introduced important methods for solving simultaneous polynomial equations in many unknowns. The most 
well-known result in this book is now called Bézout’s theorem, which in its general form tells us that the number of common points on 
two plane algebraic curves equals the product of the degrees of these curves. Bézout is also credited with inventing the determinant 
(which was called the Bézoutian by the great English mathematician James Joseph Sylvester). He was considered to be a kind person 
with a warm heart, although he had a reserved and somber personality. He was happily married and a father. 
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DEFINITION 6 


EXAMPLE 17 


If a and D are positive integers, then integers s and ¢ such that gcd(a, b) = sa + tb are called 
Bézout coefficients of a and b (after Etienne Bézout, a French mathematician of the eighteenth 
century). Also, the equation gcd(a, b) = sa + tb is called Bézout’s identity. 


We will not give a formal proof of Theorem 6 here (see Exercise 36 in Section 5.2 and [Ro10] 
for proofs). We will provide an example of a general method that can be used to find a linear 
combination of two integers equal to their greatest common divisor. (In this section, we will 
assume that a linear combination has integer coefficients.) The method proceeds by working 
backward through the divisions of the Euclidean algorithm, so this method requires a forward 
pass and a backward pass through the steps of the Euclidean algorithm. (In the exercises we 
will describe an algorithm called the extended Euclidean algorithm, which can be used to 
express gcd(a, b) as a linear combination of a and b using a single pass through the steps of the 
Euclidean algorithm; see the preamble to Exercise 41.) 


Express gced(252, 198) = 18 as a linear combination of 252 and 198. 
Solution: To show that gced(252, 198) = 18, the Euclidean algorithm uses these divisions: 


252 = 1-198 + 54 
198 = 3 - 54+ 36 
54= 1-364 18 
36=2- 18. 


Using the next-to-last division (the third division), we can express gcd(252, 198) = 18 as a 
linear combination of 54 and 36. We find that 


18 = 54—1- 36. 
The second division tells us that 
36 = 198 — 3 - 54. 


Substituting this expression for 36 into the previous equation, we can express 18 as a linear 
combination of 54 and 198. We have 


18=54—1-36=54—1-(198 —3-54)=4-54—1.- 198. 
The first division tells us that 
54 = 252 —1- 198. 


Substituting this expression for 54 into the previous equation, we can express 18 as a linear 
combination of 252 and 198. We conclude that 


18 =4- (252 —1- 198) — 1-198 =4.- 252 —5.- 198, 
completing the solution. < 
We will use Theorem 6 to develop several useful results. One of our goals will be to prove 
the part of the fundamental theorem of arithmetic asserting that a positive integer has at most 


one prime factorization. We will show that if a positive integer has a factorization into primes, 
where the primes are written in nondecreasing order, then this factorization is unique. 


LEMMA 2 


LEMMA 3 


EXAMPLE 18 
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First, we need to develop some results about divisibility. 


If a, b, and c are positive integers such that gcd(a, b) = 1 anda | bc, thena | c. 


Proof: Because gcd(a, b) = 1, by Bézout’s theorem there are integers s and t such that 
sa+tb=1. 

Multiplying both sides of this equation by c, we obtain 
sac+tbc=c. 


We can now use Theorem | of Section 4.1 to show that a | c. By part (ii) of that theorem, a | thc. 
Because a | sac and a | tbc, by part (7) of that theorem, we conclude that a divides sac + the. 
Because sac + thc = c, we conclude that a | c, completing the proof. | 


We will use the following generalization of Lemma 2 in the proof of uniqueness of prime 
factorizations. (The proof of Lemma 3 is left as Exercise 64 in Section 5.1, because it can be 
most easily carried out using the method of mathematical induction, covered in that section.) 


If p is a prime and p | a,a2---d,, where each q; is an integer, then p | a; for some 7. 


We can now show that a factorization of an integer into primes is unique. That is, we will 
show that every integer can be written as the product of primes in nondecreasing order in at 
most one way. This is part of the fundamental theorem of arithmetic. We will prove the other 
part, that every integer has a factorization into primes, in Section 5.2. 


Proof (of the uniqueness of the prime factorization of a positive integer): We will use a 
proof by contradiction. Suppose that the positive integer n can be written as the product of primes 
in two different ways, say,n = pi p2--- Ps andn = q1q2--- qr, each p; and q; are primes such 
that p) < pp <---< psandqi <q7 <--- <4. 

When we remove all common primes from the two factorizations, we have 


Pi, Pir - * Pi, = Fj jo “" Fiys 


where no prime occurs on both sides of this equation and uw and v are positive integers. By 
Lemma 3 it follows that p;, divides qj, for some k. Because no prime divides another prime, 
this is impossible. Consequently, there can be at most one factorization of n into primes in 
nondecreasing order. a4 


Lemma 2 can also be used to prove a result about dividing both sides of a congruence by 
the same integer. We have shown (Theorem 5 in Section 4.1) that we can multiply both sides of 
a congruence by the same integer. However, dividing both sides of a congruence by an integer 
does not always produce a valid congruence, as Example 18 shows. 


The congruence 14 = 8 (mod 6) holds, but both sides of this congruence cannot be divided by 2 
to produce a valid congruence because 14/2 = 7 and 8/2 = 4, but 7 ¥ 4 (mod 6). 
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Although we cannot divide both sides of a congruence by any integer to produce a valid 
congruence, we can if this integer is relatively prime to the modulus. Theorem 7 establishes this 
important fact. We use Lemma 2 in the proof. 


Let m be a positive integer and let a, b, and c be integers. If ac = bc (mod m) and 


Proof: Because ac=bc(modm), m|ac—be=c(a—b). By Lemma 2, because 
gcd(c,m) = 1, it follows that m | a — b. We conclude that a = b (mod m). 4] 


14. 


15. 


16. 


17. 


18. 


THEOREM 7 
gcd(c,m) = 1, thena = b (modm). 
Exercises 
1. Determine whether each of these integers is prime. 
a) 21 b) 29 
c) 71 d) 97 
e) 111 f) 143 
2. Determine whether each of these integers is prime. 
a) 19 b) 27 
c) 93 d) 101 
e) 107 f) 113 
3. Find the prime factorization of each of these integers. 
a) 88 b) 126 c) 729 
d) 1001 e) 1111 f) 909,090 
4. Find the prime factorization of each of these integers. 
a) 39 b) 81 c) 101 
d) 143 e) 289 f) 899 


10. 


*11. 


12. 


* 13. 


. Find the prime factorization of 10!. 
. How many zeros are there at the end of 100!? 
. Express in pseudocode the trial division algorithm for 


determining whether an integer is prime. 


. Express in pseudocode the algorithm described in the text 


for finding the prime factorization of an integer. 


. Show that if a” + 1 is composite if a and m are integers 


greater than | and m is odd. [Hint: Show that x + 1 isa 
factor of the polynomial x” + 1 if m is odd.] 

Show that if 2”-+ 1 is an odd prime, then m = 2” 
for some nonnegative integer n. [Hint: First show that 
the polynomial identity x” +1 = (x + 1I)akeOD — 


xk-2) 4... 444.1) holds, where m=kt and t 
is odd.] 
Show that log, 3 is an irrational number. Recall that an ir- 


rational number is a real number x that cannot be written 
as the ratio of two integers. 

Prove that for every positive integer n, there are n con- 
secutive composite integers. [Hint: Consider the n con- 
secutive integers starting with (n + 1)! + 2.] 

Prove or disprove that there are three consecutive odd 
positive integers that are primes, that is, odd primes of 
the form p, p + 2, and p+ 4. 


19. 


20. 


Which positive integers less than 12 are relatively prime 
to 12? 

Which positive integers less than 30 are relatively prime 
to 30? 

Determine whether the integers in each of these sets are 
pairwise relatively prime. 
a) 21, 34,55 

c) 25, 41, 49, 64 


b) 14, 17, 85 
d) 17, 18, 19, 23 


Determine whether the integers in each of these sets are 
pairwise relatively prime. 

a) 11, 15,19 b) 14, 15, 21 

ce) 12, 17,31, 37 d) 7, 8,9, 11 


We call a positive integer perfect if it equals the sum of 

its positive divisors other than itself. 

a) Show that 6 and 28 are perfect. 

b) Show that 2?-!(2? — 1) is a perfect number when 
2? — 1 is prime. 

Show that if 2” — 1 is prime, then n is prime. [Hint: Use 

the identity 27 —1 = (24 —1)- (22@-) 4. 240-2) 4. 

+++ +24 1).] 


Determine whether each of these integers is prime, veri- 
fying some of Mersenne’s claims. 

a) 27-1 b) 2°-1 

ce) 2-1 d) 245-1 


The value of the Euler ¢-function at the positive integer n 
is defined to be the number of positive integers less than or 
equal to n that are relatively prime to n. [Note: ¢ is the Greek 
letter phi.] 


21. 


22. 
23. 


24, 


Find these values of the Euler ¢-function. 

a) (4). b) #(10). c) (13). 

Show that n is prime if and only if (7) =n — 1. 

What is the value of ¢(p*) when p is prime and k is a 
positive integer? 

What are the greatest common divisors of these pairs of 
integers? 

a) 2? <3? «$2.9? 5? 

b) 2-3-5-7 + 11+13,2".3? 11-17" 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


* 36. 


Pa *37, 


38. 


39. 


ey T7179" d) 27.7, 57-13 

e) 0,5 f) 2-3-5-7,2-3-5-7 
What are the greatest common divisors of these pairs of 
integers? 

a) 3) Se TQ as ee 

b) 11+13+17,27-37 --7P 

ce) 237",23)* 

d) 41-43-53, 41-43-53 

e) 313 . 517 gl2 . 7 

f) 1111,0 

What is the least common multiple of each pair in Exer- 
cise 24? 

What is the least common multiple of each pair in Exer- 
cise 25? 

Find gcd(1000, 625) and Icm(1000, 625) and verify that 
gced(1000, 625) - Icem(1000, 625) = 1000 - 625. 

Find gced(92928, 123552) and Iem(92928, 123552), and 
verify that gcd(92928, 123552) - lem(92928, 123552) = 
92928 - 123552. [Hint: First find the prime factorizations 
of 92928 and 123552.] 

If the product of two integers is 273°577!! and their great- 
est common divisor is 23 345, what is their least common 
multiple? 

Show that if a and b are positive integers, then ab = 
gcd(a, b) - lcm(a, b). [Hint: Use the prime factorizations 
of a and b and the formulae for gcd(a, b) and Icm(a, b) 
in terms of these factorizations.] 

Use the Euclidean algorithm to find 


a) gcd(1, 5). b) gcd(100, 101). 

ce) gced(123, 277). d) gcd(1529, 14039). 
e) gcd(1529, 14038). f) gced(11111, 111111). 
Use the Euclidean algorithm to find 

a) gcd(12, 18). b) gcd(111, 201). 

ec) gced(1001, 1331). d) gcd(12345, 54321). 
e) gcd(1000, 5040). f) gcd(9888, 6060). 


How many divisions are required to find gcd(21, 34) us- 
ing the Euclidean algorithm? 

How many divisions are required to find gcd(34, 55) us- 
ing the Euclidean algorithm? 

Show that if a and b are both positive integers, then 
(24 — 1) mod (2? — 1) = 24 mod > _ 1. 

Use Exercise 36 to show that if a and b are posi- 
tive integers, then gced(2% — 1, 2? — 1) = 284.4) _ 1, 
[Hint: Show that the remainders obtained when the Eu- 
clidean algorithm is used to compute ged(2¢ — 1, 2” — 1) 
are of the form 2” — 1, where r is a remainder arising 
when the Euclidean algorithm is used to find gcd(a, b).] 
Use Exercise 37 to show that the integers 3a: 1, 4 
1, 233 — 1, 23! — 1, 229-1, and 273 — 1 are pairwise 
relatively prime. 

Using the method followed in Example 17, express the 
greatest common divisor of each of these pairs of integers 
as a linear combination of these integers. 

a) 10, 11 b) 21, 44 c) 36, 48 

d) 34,55 e) 117,213 f) 0,223 

g) 123, 2347 h) 3454,4666 ~=—i)-«9999, 11111 
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40. Using the method followed in Example 17, express the 
greatest common divisor of each of these pairs of integers 
as a linear combination of these integers. 

a) 9, 11 b) 33, 44 c) 35, 78 
d) 21,55 e) 101, 203 f) 124, 323 
g) 2002, 2339 hh) 3457,4669 ~—i):=‘:10001, 13422 


The extended Euclidean algorithm can be used to express 
gcd(a, b) as a linear combination with integer coefficients of 
the integers a and b. We set sg = 1, 5; = 0,f9 = 0, andt, = 1 
and let Sj = Sj-2 — Qj-1Sj-1 and tj = tj-2 — qj-itj-1 for 
j =2,3,...,n, where the qj; are the quotients in the di- 
visions used when the Euclidean algorithm finds gcd(a, b), 
as shown in the text. It can be shown (see [Rol0]) that 
gcd(a, b) = s,a + t,b. The main advantage of the extended 
Euclidean algorithm is that it uses one pass through the steps 
of the Euclidean algorithm to find Bézout coefficients of a 
and b, unlike the method in the text which uses two passes. 
41. Use the extended Euclidean algorithm to express 
gcd(26, 91) as a linear combination of 26 and 91. 
42. Use the extended Euclidean algorithm to express 
gcd(252, 356) as a linear combination of 252 and 356. 
43. Use the extended Euclidean algorithm to express 
gcd(144, 89) as a linear combination of 144 and 89. 
44. Use the extended Euclidean algorithm to express 
gced(1001, 100001) as a linear combination of 1001 and 
100001. 


45. Describe the extended Euclidean algorithm using pseu- 
docode. 


46. Find the smallest positive integer with exactly n different 
positive factors when n is 
a) 3. b) 4. c) 5. 
d) 6. e) 10. 

47. Can you find a formula or rule for the nth term of a se- 
quence related to the prime numbers or prime factoriza- 
tions so that the initial terms of the sequence have these 


values? 

a) 0,1, 1,0, 1,0, 1,0,0,0, 1,0, 1, 

b) 1, 2,3, 2,5, 2,7, 2,3, 2, 11,2, 3. 2; 

c) 1,2,2,3,2,4, 2, 4, 3, 4, 2, 6, 2, 4, 

d) 1,1,1,0,1,1,1,0,0,1,1,0,1,1,. 

e) 1,2,3,3,5,5,7, 7,7, 7, 11, 11, 13, B.. 

f) 1,2, 6, 30, 210, 2310, 30030, 510510, 9699690, 


223092870, ... 
48. Can you find a formula or rule for the nth term of a se- 
quence related to the prime numbers or prime factoriza- 
tions so that the initial terms of the sequence have these 


values? 

a) 2,2,3,5,5,7,7, 11, 11, 11, 11, 13, 13,. 

b) 0,1, 2,2,3,3,4,4,4,4,5,5,6,6,.. 

c) 1,0,0,1,0,1,0,1,1,1,0,1,0,1,. 

d) 1, —1, —1, 0, —1, 1, —1, 0, 0, 1, — LO-L LL. 
e) 1,1,1,1,1,0,1,1, 1,0, 1,0, 1,0, 0,. 


f) 4,9, 25, 49, 121, 169, 289, 361, 529, 841, 961, 1369,... 


49. Prove that the product of any three consecutive integers 
is divisible by 6. 
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50 


*51 


52 


53. 


54, 


55. 


. Show that if a, b, and m are integers such that m > 2 and 
a = b (mod m), then gcd(a, m) = gcd(b, m). 

. Prove or disprove that n? — 79n + 1601 is prime when- 
ever n is a positive integer. 

. Prove or disprove that pj p2--- Py + 1 is prime for every 

positive integer n, where p, p2,..., Pn are the n small- 

est prime numbers. 

Show that there is a composite integer in every arithmetic 

progression ak + b,k = 1,2,... where a and b are pos- 

itive integers. 

Adapt the proof in the text that there are infinitely many 

primes to prove that there are infinitely many primes 

of the form 3k+2, where k is a nonnegative inte- 

ger. [Hint: Suppose that there are only finitely many 

such primes g1,q92,.--,9n, and consider the number 

3q192°** dn — 1] 

Adapt the proof in the text that there are infinitely many 

primes to prove that there are infinitely many primes 
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* 56. 


* 57, 


of the form 4k+3, where k is a nonnegative inte- 
ger. [Hint: Suppose that there are only finitely many 
such primes g1,q2,.--;9n, and consider the number 


4q19q2°*: Qn—1.] 


Prove that the set of positive rational numbers is countable 
by setting up a function that assigns to a rational num- 
ber p/q with gcd(p, q) = 1 the base 11 number formed 
by the decimal representation of p followed by the base 
11 digit A, which corresponds to the decimal number 10, 
followed by the decimal representation of q. 


Prove that the set of positive rational numbers is countable 
by showing that the function K is a one-to-one correspon- 
dence between the set of positive rational numbers and 


the set of positive integers if K(m/n) = p>" ps” 


P, Po 
p™ ae ill ee ae, where gcd(m,n) = 1 


and the prime-power factorizations of m and n are m = 
eat aiana te e. andn = gy age 


Introduction 


Solving linear congruences, which have the form ax = b (mod m), is an essential task in the 
study of number theory and its applications, just as solving linear equations plays an important 
role in calculus and linear algebra. To solve linear congruences, we employ inverses modulo m. 
We explain how to work backwards through the steps of the Euclidean algorithm to find inverses 
modulo m. Once we have found an inverse of a modulo m, we solve the congruence ax = b 
(mod m) by multiplying both sides of the congruence by this inverse. 


Simultaneous systems of linear congruence have been studied since ancient times. For 
example, the Chinese mathematician Sun-Tsu studied them in the first century. We will show 
how to solve systems of linear congruences modulo pairwise relatively prime moduli. The result 
we will prove is called the Chinese remainder theorem, and our proof will give a method to 
find all solutions of such systems of congruences. We will also show how to use the Chinese 
remainder theorem as a basis for performing arithmetic with large integers. 


We will introduce a useful result of Fermat, known as Fermat’s little theorem, which states 
that if p is prime and p does not divide a, thena?~' = 1 (mod p). We will examine the converse 
of this statement, which will lead us to the concept of a pseudoprime. A pseudoprime m to the base 
a is acomposite integer m that masquerades as a prime by satisfying the congruence a”! = 1 
(mod m). We will also give an example of a Carmichael number, which is a composite integer 
that is a pseudoprime to all bases a relatively prime to it. 


We also introduce the notion of discrete logarithms, which are analogous to ordinary loga- 
rithms. To define discrete logarithms we must first define primitive roots. A primitive root of a 
prime p is an integer r such that every integer not divisible by p is congruent to a power of r 
modulo p. If r is a primitive root of p and r® = a (mod p), then e is the discrete logarithm of a 
modulo p to the base r. Finding discrete logarithms turns out to be an extremely difficult prob- 
lem in general. The difficulty of this problem is the basis for the security of many cryptographic 


systems. 
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Linear Congruences 


A congruence of the form 
ax = b(modm), 


where m is a positive integer, a and b are integers, and x is a variable, is called a linear 
congruence. Such congruences arise throughout number theory and its applications. 

How can we solve the linear congruence ax = b (mod m), that is, how can we find all 
integers x that satisfy this congruence? One method that we will describe uses an integer @ 
such that aa = | (mod m), if such an integer exists. Such an integer @ is said to be an inverse 
of a modulo m. Theorem | guarantees that an inverse of a modulo m exists whenever a and m 
are relatively prime. 


If a and m are relatively prime integers and m > 1, then an inverse of a modulo m exists. 
Furthermore, this inverse is unique modulo m. (That is, there is a unique positive integer @ 
less than m that is an inverse of a modulo m and every other inverse of a modulo m is 
congruent to a modulo m.) 


Proof: By Theorem 6 of Section 4.3, because gcd(a, m) = 1, there are integers s and t such 
that 


sat+titm=1. 

This implies that 
sa+tm=1(modm). 

Because tm = 0 (mod m), it follows that 
sa =1(modm). 


Consequently, s is an inverse of a modulo m. That this inverse is unique modulo m is left as 
Exercise 7. | 


Using inspection to find an inverse of a modulo m is easy when m is small. To find this 
inverse, we look for a multiple of a that exceeds a multiple of m by |. For example, to find an 
inverse of 3 modulo 7, we can find j -3 for 7 = 1, 2,..., 6, stopping when we find a multiple 
of 3 that is one more than a multiple of 7. We can speed this approach up if we note that 
2-3 = -—1(mod7). This means that (—2) - 3 = 1 (mod 7). Hence, 5 - 3 = 1 (mod 7), so 5 is an 
inverse of 3 modulo 7. 

We can design a more efficient algorithm than brute force to find an inverse of a modulo m 
when gcd(a, m) = | using the steps of the Euclidean algorithm. By reversing these steps as 
in Example 17 of Section 4.3, we can find a linear combination sa + tm = 1 where s and t 
are integers. Reducing both sides of this equation modulo m tells us that s is an inverse of 
a modulo m. We illustrate this procedure in Example 1. 
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EXAMPLE 2 


Find an inverse of 3 modulo 7 by first finding Bézout coefficients of 3 and 7. (Note that we have 
already shown that 5 is an inverse of 3 modulo 7 by inspection.) 


Solution: Because gcd(3, 7) = 1, Theorem | tells us that an inverse of 3 modulo 7 exists. The 
Euclidean algorithm ends quickly when used to find the greatest common divisor of 3 and 7: 


7=2-34+1. 
From this equation we see that 
—2-341-7=1. 


This shows that —2 and | are Bézout coefficients of 3 and 7. We see that —2 is an inverse of 3 
modulo 7. Note that every integer congruent to —2 modulo 7 is also an inverse of 3, such as 5, 
—9, 12, and so on. < 


Find an inverse of 101 modulo 4620. 


Solution: For completeness, we present all steps used to compute an inverse of 101 modulo 4620. 
(Only the last step goes beyond methods developed in Section 4.3 and illustrated in Example 17 
in that section.) First, we use the Euclidean algorithm to show that gcd(101, 4620) = 1. Then 
we will reverse the steps to find Bézout coefficients a and b such that 10la + 4620b = 1. It will 
then follow that a is an inverse of 101 modulo 4620. The steps used by the Euclidean algorithm 
to find gcd(101, 4620) are 


4620 = 45-1014 75 
101 =1-75+ 26 


75 =2-26+ 23 
26=1-23+3 
23 =7-34+2 
3=1-2+1 
2=2-1. 


Because the last nonzero remainder is 1, we know that gcd(101, 4620) = 1. We can now find 
the Bézout coefficients for 101 and 4620 by working backwards through these steps, expressing 
gcd(101, 4620) = | in terms of each successive pair of remainders. In each step we eliminate 
the remainder by expressing it as a linear combination of the divisor and the dividend. We obtain 
1=3-1.-2 

=3-1-(23-—7-3)=-1-2348.-3 

=—1-2348-(Q6—1-23) =8-26—9.- 23 

=8-.26-—9.(75 —2-26) = —9- 75 + 26-26 

= —9-75+ 26- (101 — 1-75) = 26-101 — 35-75 

= 26-101 — 35 - (4620 — 45 - 101) = —35 - 4620+ 1601 - 101. 


That —35 - 4620 + 1601 - 101 = 1 tells us that —35 and 1601 are Bézout coefficients of 4620 
and 101, and 1601 is an inverse of 101 modulo 4620. < 


EXAMPLE 3 


Links 


EXAMPLE 4 
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Once we have an inverse a of a modulo m, we can solve the congruence ax = b (mod m) 
by multiplying both sides of the linear congruence by a, as Example 3 illustrates. 


What are the solutions of the linear congruence 3x = 4 (mod 7)? 


Solution: By Example | we know that —2 is an inverse of 3 modulo 7. Multiplying both sides 
of the congruence by —2 shows that 


—2-3x =—2-4 (mod 7). 


Because —6 = | (mod 7) and —8 = 6 (mod 7), it follows that if x is a solution, then x = —8 = 
6 (mod 7). 

We need to determine whether every x with x = 6 (mod 7) is a solution. Assume that 
x = 6(mod 7). Then, by Theorem 5 of Section 4.1, it follows that 


3x =3-6= 18 =4(mod 7), 


which shows that all such x satisfy the congruence. We conclude that the solutions to the 
congruence are the integers x such that x = 6(mod 7), namely, 6, 13, 20,... and —1, —8, 
—15,.... 


The Chinese Remainder Theorem 


Systems of linear congruences arise in many contexts. For example, as we will see later, they are 
the basis for a method that can be used to perform arithmetic with large integers. Such systems 
can even be found as word puzzles in the writings of ancient Chinese and Hindu mathematicians, 
such as that given in Example 4. 


In the first century, the Chinese mathematician Sun-Tsu asked: 


There are certain things whose number is unknown. When divided by 3, the remainder 
is 2; when divided by 5, the remainder is 3; and when divided by 7, the remainder is 2. What 
will be the number of things? 


This puzzle can be translated into the following question: What are the solutions of the 
systems of congruences 


x =2 (mod 3), 
x =3/(mod 5), 
x =2 (mod 7)? 
We will solve this system, and with it Sun-Tsu’s puzzle, later in this section. < 


The Chinese remainder theorem, named after the Chinese heritage of problems involving 
systems of linear congruences, states that when the moduli of a system of linear congruences 
are pairwise relatively prime, there is a unique solution of the system modulo the product of the 
moduli. 
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EXAMPLE 5 


THE CHINESE REMAINDER THEOREM Let my, m2, ..., my, be pairwise relatively 
prime positive integers greater than one and a1, a2, ... , a, arbitrary integers. Then the system 


a, (mod m1), 
xX = a2 (mod m2), 


xX = dy, (mod my) 


has a unique solution modulo m = m,m2---m,. (That is, there is a solution x with 
0 < x < m, and all other solutions are congruent modulo m to this solution.) 


Proof: To establish this theorem, we need to show that a solution exists and that it is unique 
modulo m. We will show that a solution exists by describing a way to construct this solution; 
showing that the solution is unique modulo m is Exercise 30. 

To construct a simultaneous solution, first let 


Mz = m/mx 


fork = 1,2,...,n.Thatis, M; is the product of the moduli except for mz. Because m; and mx 
have no common factors greater than 1 when i # k, it follows that gcd(m,;, My) = 1. Conse- 
quently, by Theorem 1, we know that there is an integer y;,, an inverse of M; modulo m,, such 
that 


Myyz = 1 (mod mx). 
To construct a simultaneous solution, form the sum 
x =a)M\y, + a2Mzy2 + +++ + anMnyn. 


We will now show that x is a simultaneous solution. First, note that because M; = 0 (mod mx) 
whenever j # k, all terms except the kth term in this sum are congruent to 0 modulo m,;. Because 
Myr = | (mod mx) we see that 


x = agMryr = ag (mod mg), 
fork = 1,2,...,n. We have shown that x is a simultaneous solution to the n congruences. <I 


Example 5 illustrates how to use the construction given in our proof of the Chinese remainder 
theorem to solve a system of congruences. We will solve the system given in Example 4, arising 
in Sun-Tsu’s puzzle. 


To solve the system of congruences in Example 4, first let m = 3-5-7 = 105, Mj =m/3= 
35, M2 = m/5 = 21, and M3 = m/7 = 15. We see that 2 is an inverse of M; = 35 modulo 3, 
because 35-2 =2-2=1 (mod 3); | is an inverse of Mz = 21 modulo 5, because 21 = 
1 (mod 5); and | is an inverse of M3 = 15 (mod 7), because 15 = | (mod 7). The solutions to 
this system are those x such that 


x =a,M,y, + a2Moy2 + a3M3y3 =2-35-243-21-142-15-1 
= 233 = 23 (mod 105). 


EXAMPLE 6 


EXAMPLE 7 
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It follows that 23 is the smallest positive integer that is a simultaneous solution. We conclude that 
23 is the smallest positive integer that leaves a remainder of 2 when divided by 3, a remainder 
of 3 when divided by 5, and a remainder of 2 when divided by 7. 


Although the construction in Theorem 2 provides a general method for solving systems of 
linear congruences with pairwise relatively prime moduli, it can be easier to solve a system using 
a different method. Example 6 illustrates the use of a method known as back substitution. 


Use the method of back substitution to find all integers x such that x = 1 (mod 5), 
x =2 (mod 6), and x =3 (mod 7). 


Solution: By Theorem 4 in Section 4.1, the first congruence can be rewritten as an equality, 
x = 5t + 1 where f is an integer. Substituting this expression for x into the second congruence 
tells us that 


5t + 1 =2(mod6), 


which can be easily solved to show that t = 5 (mod 6) (as the reader should verify). Using 
Theorem 4 in Section 4.1 again, we see that t = 6u + 5 where u is an integer. Substituting this 
expression for ¢ back into the equation x = 5f + | tells us that x = 5(6u + 5) + 1 = 30u + 26. 
We insert this into the third equation to obtain 


30u + 26 = 3 (mod 7). 


Solving this congruence tells us that uv = 6 (mod 7) (as the reader should verify). Hence, Theo- 
rem 4 in Section 4.1 tells us that 1 = 7v + 6 where v is an integer. Substituting this expression 
for u into the equation x = 30u + 26 tells us that x = 30(7v + 6) + 26 = 210u + 206. Trans- 
lating this back into a congruence, we find the solution to the simultaneous congruences, 


x = 206 (mod 210). < 


Computer Arithmetic with Large Integers 


Suppose that m1, m2, ..., Mp, are pairwise relatively prime moduli and let m be their product. 
By the Chinese remainder theorem, we can show (see Exercise 28) that an integer a with 
0 <a < mcanbe uniquely represented by the n-tuple consisting of its remainders upon division 
by m;,i = 1,2,...,n. That is, we can uniquely represent a by 


(a mod m,,a mod m2, ...,a mod m,). 


What are the pairs used to represent the nonnegative integers less than 12 when they are rep- 
resented by the ordered pair where the first component is the remainder of the integer upon 
division by 3 and the second component is the remainder of the integer upon division by 4? 


Solution: We have the following representations, obtained by finding the remainder of each 
integer when it is divided by 3 and by 4: 


0=(0,0) 4=(,0) 8=(,0) 
i=l Sah G=0-1 
=D 63 0,2). 10S 0,9) 
5 = (0,3) 7,3) - 11 = 2.3). 
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To perform arithmetic with large integers, we select moduli m,, m2, ..., mn, where each m; 
is an integer greater than 2, gcd(m;, mj) = 1 wheneveri ¢ j, and m = m,mz--- mp is greater 
than the results of the arithmetic operations we want to carry out. 

Once we have selected our moduli, we carry out arithmetic operations with large integers by 
performing componentwise operations on the n-tuples representing these integers using their 
remainders upon division by m;,i = 1,2,...,n. Once we have computed the value of each 
component in the result, we recover its value by solving a system of n congruences modulo 
m;,i = 1,2,...,n. This method of performing arithmetic with large integers has several valu- 
able features. First, it can be used to perform arithmetic with integers larger than can ordinarily 
be carried out on a computer. Second, computations with respect to the different moduli can be 
done in parallel, speeding up the arithmetic. 


Suppose that performing arithmetic with integers less than 100 on a certain processor is much 
quicker than doing arithmetic with larger integers. We can restrict almost all our computations to 
integers less than 100 if we represent integers using their remainders modulo pairwise relatively 
prime integers less than 100. For example, we can use the moduli of 99, 98, 97, and 95. (These 
integers are relatively prime pairwise, because no two have a common factor greater than 1.) 

By the Chinese remainder theorem, every nonnegative integer less than 99 - 98 - 97-95 = 
89,403,930 can be represented uniquely by its remainders when divided by these four mod- 
uli. For example, we represent 123,684 as (33, 8, 9, 89), because 123,684 mod 99 = 33; 
123,684 mod 98 = 8; 123,684 mod 97 = 9; and 123,684 mod 95 = 89. Similarly, we represent 
413,456 as (32, 92, 42, 16). 

To find the sum of 123,684 and 413,456, we work with these 4-tuples instead of these two 
integers directly. We add the 4-tuples componentwise and reduce each component with respect 
to the appropriate modulus. This yields 


(33, 8, 9, 89) + (32, 92, 42, 16) 
= (65 mod 99, 100 mod 98, 51 mod 97, 105 mod 95) 
= (65, 2, 51, 10). 


To find the sum, that is, the integer represented by (65, 2, 51, 10), we need to solve the 
system of congruences 


= 65 (mod 99), 
= 2 (mod 98), 
= 51 (mod 97), 
= 10 (mod 95). 


SB Se BS 


It can be shown (see Exercise 53) that 537,140 is the unique nonnegative solution of this 
system less than 89,403,930. Consequently, 537,140 is the sum. Note that it is only when we 
have to recover the integer represented by (65, 2, 51, 10) that we have to do arithmetic with 
integers larger than 100. 


Particularly good choices for moduli for arithmetic with large integers are sets of integers of 
the form 2* — 1, where k is a positive integer, because it is easy to do binary arithmetic modulo 
such integers, and because it is easy to find sets of such integers that are pairwise relatively prime. 
[The second reason is a consequence of the fact that gcd(2° — 1, 25 — 1) = 28°d(@,b) __ 1 ag 
Exercise 37 in Section 4.3 shows.] Suppose, for instance, that we can do arithmetic with integers 
less than 23° easily on our computer, but that working with larger integers requires special 
procedures. We can use pairwise relatively prime moduli less than 2° to perform arithmetic 
with integers as large as their product. For example, as Exercise 38 in Section 4.3 shows, the 
integers 25 — 1, 234 — 1, 233 — 1, 23! — 1, 279 — 1, and 273 — 1 are pairwise relatively prime. 
Because the product of these six moduli exceeds 2!*4, we can perform arithmetic with integers 
as large as 2!*4 (as long as the results do not exceed this number) by doing arithmetic modulo 
each of these six moduli, none of which exceeds 2. 
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Fermat’s Little Theorem 


The great French mathematician Pierre de Fermat made many important discoveries in number 
theory. One of the most useful of these states that p divides a?—! — 1 whenever p is prime 
and a is an integer not divisible by p. Fermat announced this result in a letter to one of his 
correspondents. However, he did not include a proof in the letter, stating that he feared the proof 
would be too long. Although Fermat never published a proof of this fact, there is little doubt that 
he knew how to prove it, unlike the result known as Fermat’s last theorem. The first published 
proof is credited to Leonhard Euler. We now state this theorem in terms of congruences. 


FERMAT?’S LITTLE THEOREM If p is prime and a is an integer not divisible by p, 
then 


a?—! = 1 (mod p). 
Furthermore, for every integer a we have 


a? =a (mod p). 


Remark: Fermat’s little theorem tells us that if a € Zp, then a? l_1inZ p 


The proof of Theorem 3 is outlined in Exercise 19. 
Fermat’s little theorem is extremely useful in computing the remainders modulo p of large 
powers of integers, as Example 9 illustrates. 


Find 7222 mod 11. 


Solution: We can use Fermat’s little theorem to evaluate 7*7* mod 11 rather than using the fast 
modular exponentiation algorithm. By Fermat’s little theorem we know that 7!° = 1 (mod 11), 
so (7!°)k = 1 (mod 11) for every positive integer k. To take advantage of this last congruence, 
we divide the exponent 222 by 10, finding that 222 = 22 - 10 + 2. We now see that 


7222 = 7221042 = (oer = (1)? 49 =5 (mod 11). 
It follows that 777 mod 11 =5. < 


Example 9 illustrated how we can use Fermat’s little theorem to compute a” mod p, where 
p is prime and p J a. First, we use the division algorithm to find the quotient g and remainder 
r when n is divided by p — I, so that n = g(p — 1) +r where 0 <r < p — 1. It follows that 
a” = qi(P—)+r — (qP—!)1q" = 14a" =a" (mod p). Hence, to find a” mod p, we only need 
to compute a’ mod p. We will take advantage of this simplification many times in our study of 
number theory. 


Pseudoprimes 


In Section 4.2 we showed that an integer n is prime when it is not divisible by any prime p with 
p < Jn. Unfortunately, using this criterion to show that a given integer is prime is inefficient. 
It requires that we find all primes not exceeding ./n and that we carry out trial division by each 
such prime to see whether it divides n. 
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DEFINITION 1 


Are there more efficient ways to determine whether an integer is prime? According to some 
sources, ancient Chinese mathematicians believed that n was an odd prime if and only if 


2"-! = | (modn). 


If this were true, it would provide an efficient primality test. Why did they believe this 
congruence could be used to determine whether an integer n > 2 is prime? First, they observed 
that the congruence holds whenever n is an odd prime. For example, 5 is prime and 


2>-! = 2* = 16 = 1 (mod 5). 
By Fermat’s little theorem, we know that this observation was correct, that is, gn-l — 
1 (mod n) whenever n is an odd prime. Second, they never found a composite integer n for 
which the congruence holds. However, the ancient Chinese were only partially correct. They 
were correct in thinking that the congruence holds whenever n is prime, but they were incorrect 
in concluding that n is necessarily prime if the congruence holds. 

Unfortunately, there are composite integers n such that 2”~! = 1 (mod n). Such integers 
are called pseudoprimes to the base 2. 


The integer 341 is a pseudoprime to the base 2 because it is composite (341 = 11-31) and as 
Exercise 37 shows 


2°40 = | (mod 341). < 


We can use an integer other than 2 as the base when we study pseudoprimes. 


Let b be a positive integer. If n is a composite positive integer, and b”~! = 1 (mod n), then 
n is called a pseudoprime to the base b. 


Given a positive integer n, determining whether 2”~! = 1 (mod n) is a useful test that pro- 
vides some evidence concerning whether v is prime. In particular, if n satisfies this congruence, 
then it is either prime or a pseudoprime to the base 2; if n does not satisfy this congruence, it is 
composite. We can perform similar tests using bases b other than 2 and obtain more evidence 
as to whether n is prime. If n passes all such tests, it is either prime or a pseudoprime to all the 
bases b we have chosen. Furthermore, among the positive integers not exceeding x, where x 
is a positive real number, compared to primes there are relatively few pseudoprimes to the 
base b, where b is a positive integer. For example, among the positive integers less than 10!° 
there are 455,052,512 primes, but only 14,884 pseudoprimes to the base 2. Unfortunately, we 


PIERRE DE FERMAT (1601-1665) Pierre de Fermat, one of the most important mathematicians of the 
seventeenth century, was a lawyer by profession. He is the most famous amateur mathematician in history. Fermat 
published little of his mathematical discoveries. It is through his correspondence with other mathematicians 
that we know of his work. Fermat was one of the inventors of analytic geometry and developed some of 
the fundamental ideas of calculus. Fermat, along with Pascal, gave probability theory a mathematical basis. 
Fermat formulated what was the most famous unsolved problem in mathematics. He asserted that the equation 
x” + y” = 7" has no nontrivial positive integer solutions when n is an integer greater than 2. For more than 300 
years, no proof (or counterexample) was found. In his copy of the works of the ancient Greek mathematician 
Diophantus, Fermat wrote that he had a proof but that it would not fit in the margin. Because the first proof, 


found by Andrew Wiles in 1994, relies on sophisticated, modern mathematics, most people think that Fermat thought he had a proof, 
but that the proof was incorrect. However, he may have been tempting others to look for a proof, not being able to find one himself. 
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cannot distinguish between primes and pseudoprimes just by choosing sufficiently many bases, 
because there are composite integers n that pass all tests with bases b such that gcd(b, n) = 1. 
This leads to Definition 2. 


A composite integer n that satisfies the congruence b”~! = 1 (mod 1) for all positive integers b 
with gcd(b, n) = | is called a Carmichael number. (These numbers are named after Robert 
Carmichael, who studied them in the early twentieth century.) 


The integer 561 is a Carmichael number. To see this, first note that 561 is composite be- 
cause 561 = 3-11-17. Next, note that if gcd(b, 561) = 1, then gcd(b, 3) = ged(b, 11) = 
gcd(b, 17) = 1. 

Using Fermat’s little theorem we find that 


b* = 1 (mod 3), b!° = 1 (mod 11), and b!® = 1 (mod 17). 
It follows that 


5°60 — (p7)780 = | (mod 3), 
5°69 — (b!9)56 = | (mod 11), 
5°69 — (b!6)35 = 1 (mod 17). 


By Exercise 29, it follows that b>©? = 1 (mod 561) for all positive integers b with gcd(b, 561) = 
1. Hence 561 is a Carmichael number. < 


Although there are infinitely many Carmichael numbers, more delicate tests, described in 
the exercise set, can be devised that can be used as the basis for efficient probabilistic primality 
tests. Such tests can be used to quickly show that it is almost certainly the case that a given 
integer is prime. More precisely, if an integer is not prime, then the probability that it passes a 
series of tests is close to 0. We will describe such a test in Chapter 7 and discuss the notions 
from probability theory that this test relies on. These probabilistic primality tests can be used, 
and are used, to find large primes extremely rapidly on computers. 


Primitive Roots and Discrete Logarithms 


In the set of positive real numbers, if b > 1, and x = b’, we say that y is the logarithm of x to 
the base b. Here, we will show that we can also define the concept of logarithms modulo p of 
positive integers where p is a prime. Before we do so, we need a definition. 


A primitive root modulo a prime p is an integer r in Z, such that every nonzero element of 
Zyp is a power of r. 


ROBERT DANIEL CARMICHAEL (1879-1967) Robert Daniel Carmichael was born in Alabama. He re- 
ceived his undergraduate degree from Lineville College in 1898 and his Ph.D. in 1911 from Princeton. 
Carmichael held positions at Indiana University from 1911 until 1915 and at the University of Illinois from 
1915 until 1947. Carmichael was an active researcher in a wide variety of areas, including number theory, real 
analysis, differential equations, mathematical physics, and group theory. His Ph.D. thesis, written under the 
direction of G. D. Birkhoff, is considered the first significant American contribution to the subject of differential 
equations. 
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EXAMPLE 12 


DEFINITION 4 


EXAMPLE 13 


The discrete logarithm 
problem is hard! 


Determine whether 2 and 3 are primitive roots modulo 11. 


Solution: When we compute the powers of 2 in Z,;, we obtain 2§=2,27=4,27=8,24=5, 
P= 10, 2 = 9,27 =7, 2 = 3, Y= 6, 2!9 — 1. Because every element of Z1; is a power of 
2, 2 is a primitive root of 11. 

When we compute the powers of 3 modulo 11, we obtain 3! = 3, 32 = 9,33 = 5,34 = 4, 
3° = 1. We note that this pattern repeats when we compute higher powers of 3. Because not all 
elements of Z1; are powers of 3, we conclude that 3 is not a primitive root of 11. < 


An important fact in number theory is that there is a primitive root modulo p for every 
prime p. We refer the reader to [Ro10] for a proof of this fact. Suppose that p is prime and r is 
a primitive root modulo p. If a is an integer between | and p — 1, that is, an element of Zp, we 
know that there is an unique exponent e such that r° = a in Zp, that is, r° mod p = a. 


Suppose that p is a prime, r is a primitive root modulo p, and a is an integer between | and 
p — 1 inclusive. If r° mod p = a and 0 < e < p — 1, we say that e is the discrete logarithm 
of a modulo p to the base r and we write log, a = e (where the prime p is understood). 


Find the discrete logarithms of 3 and 5 modulo 11 to the base 2. 


Solution: When we computed the powers of 2 modulo 11 in Example 12, we found that 28 = 3 
and 2* = 5 in Z1. Hence, the discrete logarithms of 3 and 5 modulo 11 to the base 2 are 8 
and 4, respectively. (These are the powers of 2 that equal 3 and 5, respectively, in Z11.) We write 
log, 3 = 8 and log, 5 = 4 (where the modulus 11 is understood and not explicitly noted in the 
notation). 


The discrete logarithm problem takes as input a prime p, a primitive root r modulo p, 
and a positive integer a € Z); its output is the discrete logarithm of a modulo p to the base 
r. Although this problem might seem not to be that difficult, it turns out that no polynomial 
time algorithm is known for solving it. The difficulty of this problem plays an important role in 
cryptography, as we will see in Section 4.6 


Exercises 


. Show that 15 is an inverse of 7 modulo 26. 


2. Show that 937 is an inverse of 13 modulo 2436. 


. By inspection (as discussed prior to Example 1), find an 


inverse of 4 modulo 9. 


. By inspection (as discussed prior to Example 1), find an 


inverse of 2 modulo 17. 


. Find an inverse of a modulo m for each of these pairs 


of relatively prime integers using the method followed in 
Example 2. 

a)a=4,m=9 

b) a= 19,m = 141 

c) a=55,m = 89 

d) a = 89, m = 232 


. Find an inverse of a modulo m for each of these pairs 


of relatively prime integers using the method followed in 
Example 2. 

a) a=2,m=17 

b) a = 34,m = 89 


bai 


10. 


11. 


c) a= 144, m = 233 

d) a = 200, m = 1001 

Show that if a and m are relatively prime positive inte- 
gers, then the inverse of a modulo m is unique modulo 
m. [Hint: Assume that there are two solutions b and c 
of the congruence ax = 1 (mod m). Use Theorem 7 of 
Section 4.3 to show that b = c (mod m).] 


. Show that an inverse of a modulo m, where a is an in- 


teger and m > 2 is a positive integer, does not exist if 
gcd(a,m) > 1. 


. Solve the congruence 4x = 5 (mod 9) using the inverse 


of 4 modulo 9 found in part (a) of Exercise 5. 

Solve the congruence 2x = 7 (mod 17) using the inverse 
of 2 modulo 7 found in part (a) of Exercise 6. 

Solve each of these congruences using the modular in- 
verses found in parts (b), (c), and (d) of Exercise 5. 

a) 19x = 4 (mod 141) 

b) 55x = 34 (mod 89) 

c) 89x = 2 (mod 232) 


12. 


13. 


14. 


#15, 


16. 


17. 


*18. 


*19, 


Solve each of these congruences using the modular in- 
verses found in parts (b), (c), and (d) of Exercise 6. 

a) 34x = 77 (mod 89) 

b) 144x = 4 (mod 233) 

c) 200x = 13 (mod 1001) 

Find the solutions of the congruence 15x? + 19x = 5 
(mod 11). [Hint: Show the congruence is equivalent to 
the congruence 15x? + 19x + 6 = 0(mod 11). Factor the 
left-hand side of the congruence; show that a solution of 
the quadratic congruence is a solution of one of the two 
different linear congruences. ] 


Find the solutions of the congruence 12x? + 25x = 
10 (mod 11). [Hint: Show the congruence is equivalence 
to the congruence 12x? + 25x + 12 = 0 (mod 11). Fac- 
tor the left-hand side of the congruence; show that a so- 
lution of the quadratic congruence is a solution of one of 
two different linear congruences. ] 


Show that if m is an integer greater than 1 and ac = 

bc (mod m), then a = b (mod m/gcd(c, m)). 

a) Show that the positive integers less than 11, except 
1 and 10, can be split into pairs of integers such that 
each pair consists of integers that are inverses of each 
other modulo 11. 

b) Use part (a) to show that 10! = —1 (mod 11). 

Show that if p is prime, the only solutions of x? = 

1 (mod p) are integers x such that x = 1 (mod p) or 

= —1 (mod p). 

a) Generalize the result in part (a) of Exercise 16; that 
is, show that if p is a prime, the positive integers less 
than p, except 1 and p — 1, canbe split into (p — 3)/2 
pairs of integers such that each pair consists of inte- 
gers that are inverses of each other. [Hint: Use the 
result of Exercise 17.] 

b) From part (a) conclude that (p — 1)! = —1 (mod p) 
whenever p is prime. This result is known as Wilson’s 
theorem. 

c) What can we conclude if n is a positive integer such 
that (n — 1)! 4 —1 (mod n)? 

This exercise outlines a proof of Fermat’s little theorem. 

a) Suppose that a is not divisible by the prime p. Show 
that no two of the integers 1- a, 2-a,..., (p— a 
are congruent modulo p. 

b) Conclude from part (a) that the product of 
1,2,..., p — 11s congruent modulo p to the prod- 
uct of a, 2a,..., (p — la. Use this to show that 


(p — 1)! = a?~!(p — 1)! (mod p). 


c) Use Theorem 7 of Section 4.3 to show from part (b) 
that a?—! = 1 (mod p) if p J a. [Hint: Use Lemma 3 
of Section 4.3 to show that p does not divide (p — 1)! 
and then use Theorem 7 of Section 4.3. Alternatively, 
use Wilson’s theorem from Exercise 18(b).] 

d) Use part (c) to show that a? = a (mod p) for all in- 
tegers a. 


20. 


21. 


22. 


23. 


24. 


25. 


* 26. 


*27, 


28. 


*29, 


* 30. 


31. 


32. 


33. 
34. 
35. 


36. 
37. 
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Use the construction in the proof of the Chinese remainder 
theorem to find all solutions to the system of congruences 
x = 2 (mod 3), x = | (mod 4), and x = 3 (mod 5). 
Use the construction in the proof of the Chinese remain- 
der theorem to find all solutions to the system of congru- 
ences x = | (mod 2), x = 2 (mod 3), x = 3 (mod 5), and 
x =4(mod 11). 

Solve the system of congruence x = 3 (mod 6) and 
x = 4 (mod 7) using the method of back substitution. 


Solve the system of congruences in Exercise 20 using the 
method of back substitution. 


Solve the system of congruences in Exercise 21 using the 
method of back substitution. 


Write out in pseudocode an algorithm for solving a si- 
multaneous system of linear congruences based on the 
construction in the proof of the Chinese remainder theo- 
rem. 


Find all solutions, if any, to the system of congruences 
x =5 (mod 6), x = 3 (mod 10), and x = 8 (mod 15). 

Find all solutions, if any, to the system of congruences 
x =7 (mod 9), x = 4 (mod 12), and x = 16 (mod 21). 


Use the Chinese remainder theorem to show that an 
integer a, with 0 <a <m==mym2---my, where the 
positive integers m,,m2,..., My, are pairwise relatively 
prime, can be represented uniquely by the n-tuple 
(a mod m,, a mod m2, ..., a mod m,). 


Let m1, m2, ..., Mn be pairwise relatively prime integers 
greater than or equal to 2. Show that if a = b (mod m;) 
for i=1,2,...,n, then a=b(mod m), where m = 
mim2---my. (This result will be used in Exercise 30 
to prove the Chinese remainder theorem. Consequently, 
do not use the Chinese remainder theorem to prove it.) 


Complete the proof of the Chinese remainder theorem 
by showing that the simultaneous solution of a system 
of linear congruences modulo pairwise relatively prime 
moduli is unique modulo the product of these moduli. 
[Hint: Assume that x and y are two simultaneous solu- 
tions. Show that m; | x — y for all i. Using Exercise 29, 
conclude that m = mjm2---my, |x —y.] 

Which integers leave a remainder of 1 when divided by 2 
and also leave a remainder of | when divided by 3? 


Which integers are divisible by 5 but leave a remainder 

of 1 when divided by 3? 

Use Fermat’s little theorem to find 7/2! mod 13. 

Use Fermat’s little theorem to find 23!9? mod 41. 

Use Fermat’s little theorem to show that if p is prime and 

p Ja, then a?~? is an inverse of a modulo p. 

Use Exercise 35 to find an inverse of 5 modulo 41. 

a) Show that 249 = 1 (mod 11) by Fermat’s little theo- 
rem and noting that 2340 = (2!0)34, 

b) Show that 234° =1 (mod 31) using the fact that 
2340 = (2>)68 = 3268. 

c) Conclude from parts (a) and (b) that 2340 — 
1 (mod 341). 
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38. a) Use Fermat’s little theorem to compute 3302 mod 55 
3302 mod 7, and 33°? mod 11. 
b) Use your results from part (a) and the Chinese re- 
mainder theorem to find 3°? mod 385. (Note that 
385 =5-7-11.) 
39. a) Use Fermat’s little theorem to compute 52003 mod ds 
57003 mod 11, and 57° mod 13. 
b) Use your results from part (a) and the Chinese re- 
mainder theorem to find 52° mod 1001. (Note that 
1001 = 7-11-13.) 
40. Show with the help of Fermat’s little theorem that if n is 
a positive integer, then 42 divides n’ — n. 
41. Show that if p is an odd prime, then every divisor of the 
Mersenne number 2? — 1 is of the form 2kp + 1, where 
k is a nonnegative integer. [Hint: Use Fermat’s little the- 
orem and Exercise 37 of Section 4.3.] 
42. Use Exercise 41 to determine whether M)3 = 2/3 —1 = 
8191 and Mo3 = 273 — 1 = 8,388,607 are prime. 
43. Use Exercise 41 to determine whether M;, = 2!! —1 = 
2047 and Mj7 = 2!” — 1 = 131,071 are prime. 


=~ Let n be a positive integer and let n — 1 = 2°t, where s is a 


nonnegative integer and ¢ is an odd positive integer. We say 

that n passes Miller’s test for the base b if either b' = 1 (mod 

n) or b?’' = —1 (mod n) for some j with 0 < j <s—1.It 

can be shown (see [Ro10]) that a composite integer n passes 

Miller’s test for fewer than n/4 bases b with 1 <b<n.A 

composite positive integer n that passes Miller’s test to the 

base b is called a strong pseudoprime to the base D. 

* 44, Show that if 1 is prime and b is a positive integer with 
n J b, then n passes Miller’s test to the base b. 

45. Show that 2047 is a strong pseudoprime to the base 2 by 
showing that it passes Miller’s test to the base 2, but is 
composite. 

46. Show that 1729 is a Carmichael number. 

47. Show that 2821 is a Carmichael number. 

*48. Show that if n = pip2--- pe, where pj, po,..., Dk 
are distinct primes that satisfy p; —1|n— 1 for j = 
1,2,...,, then n is a Carmichael number. 

49. a) Use Exercise 48 to show that every integer of the form 
(6m + 1)(12m + 1)(18m + 1), where m is a positive 
integer and 6m + 1, 12m+ 1, and 18m + 1 are all 
primes, is a Carmichael number. 

b) Use part (a) to show that 172,947,529 is a Car- 
michael number. 

50. Find the nonnegative integer a less than 28 represented by 
each of these pairs, where each pair represents (a mod 4, 


a mod 7). 

a) (0, 0) b) (1, 0) ce) dd, 1) 
d) (2, 1) e) (2, 2) f) (0, 3) 
g) (2,0) h) G,5) i) (3,6) 


51. Express each nonnegative integer a less than 15 as a pair 
(a mod 3, a mod 5). 


52. Explain how to use the pairs found in Exercise 51 to 
add 4 and 7. 


53. Solve the system of congruences that arises in Example 8. 


54. Show that 2 is a primitive root of 19. 


55. Find the discrete logarithms of 5 and 6 to the base 2 mod- 
ulo 19. 


56. Let p be an odd prime and r a primitive root of p. 
Show that if a and b are positive integers in Z,, then 
log, (ab) = log, a + log, b (mod p — 1). 

57. Write out a table of discrete logarithms modulo 17 with 
respect to the primitive root 3. 

If m is a positive integer, the integer a is a quadratic residue 

of m if gcd(a, m) = 1 and the congruence x? =a(modm) 

has a solution. In other words, a quadratic residue of m is 
an integer relatively prime to m that is a perfect square mod- 

ulo m. If a is not a quadratic residue of m and gcd(a, m) = 1, 

we say that it is a quadratic nonresidue of m. For exam- 

ple, 2 is a quadratic residue of 7 because gced(2, 7) = 1 and 

3? = 2 (mod 7) and 3 is a quadratic nonresidue of 7 because 

gcd(3, 7) = 1 and x? =3 (mod 7) has no solution. 

58. Which integers are quadratic residues of 11? 


59. Show that if p is an odd prime and a is an integer not 
divisible by p, then the congruence x? = a (mod p) has 
either no solutions or exactly two incongruent solutions 
modulo p. 

60. Show that if p is an odd prime, then there are exactly 
(p — 1)/2 quadratic residues of p among the integers 
1,2,...,p—1. 

If p is an odd prime and a is an integer not divisible by p, the 


Legendre symbol (“) is defined to be 1 if a is a quadratic 


P 
residue of p and —1 otherwise. 


61. Show that if p is an odd prime and a and D are integers 
with a = b (mod p), then 


62. Prove Euler’s criterion, which states that if p is an odd 
prime and a is a positive integer not divisible by p, then 


(<) = a?—)/? (mod p). 
P 


(Hint: If a is a quadratic residue modulo p, apply Fer- 
mat’s little theorem; otherwise, apply Wilson’s theorem, 
given in Exercise 18(b).] 

63. Use Exercise 62 to show that if p is an odd prime and a 
and b are integers not divisible by p, then 


Gael 

P P/)\p)- 

64. Show that if p is an odd prime, then —1 is a quadratic 
residue of p if p = 1 (mod 4), and —1 is not a quadratic 
residue of p if p = 3 (mod 4). [Hint: Use Exercise 62.] 

65. Find all solutions of the congruence x? = 29 (mod 35). 


[Hint: Find the solutions of this congruence modulo 5 and 
modulo 7, and then use the Chinese remainder theorem. | 


4.5 Applications of Congruences 287 


66. Find all solutions of the congruence x2 = 16 (mod 105). 67. Describe a brute force algorithm for solving the discrete 
[Hint: Find the solutions of this congruence modulo 3, logarithm problem and find the worst-case and average- 
modulo 5, and modulo 7, and then use the Chinese re- case time complexity of this algorithm. 


mainder theorem. ] 


Applications of Congruences 


Congruences have many applications to discrete mathematics, computer science, and many 
other disciplines. We will introduce three applications in this section: the use of congruences 
to assign memory locations to computer files, the generation of pseudorandom numbers, and 
check digits. 

Suppose that a customer identification number is ten digits long. To retrieve customer files 
quickly, we do not want to assign a memory location to a customer record using the ten-digit 
identification number. Instead, we want to use a smaller integer associated to the identification 
number. This can be done using what is known as a hashing function. In this section we will 
show how we can use modular arithmetic to do hashing. 

Constructing sequences of random numbers is important for randomized algorithms, for 
simulations, and for many other purposes. Constructing a sequence of truly random numbers is 
extremely difficult, or perhaps impossible, because any method for generating what are supposed 
to be random numbers may generate numbers with hidden patterns. As a consequence, methods 
have been developed for finding sequences of numbers that have many desirable properties of 
random numbers, and which can be used for various purposes in place of random numbers. 
In this section we will show how to use congruences to generate sequences of pseudorandom 
numbers. The advantage is that the pseudorandom numbers so generated are constructed quickly; 
the disadvantage is that they have too much predictability to be used for many tasks. 

Congruences also can be used to produce check digits for identification numbers of various 
kinds, such as code numbers used to identify retail products, numbers used to identify books, 
airline ticket numbers, and so on. We will explain how to construct check digits using congru- 
ences for a variety of types of identification numbers. We will show that these check digits can 
be used to detect certain kinds of common errors made when identification numbers are printed. 


Hashing Functions 


The central computer at an insurance company maintains records for each of its customers. 
Links How can memory locations be assigned so that customer records can be retrieved quickly? The 
solution to this problem is to use a suitably chosen hashing function. Records are identified 
using a key, which uniquely identifies each customer’s records. For instance, customer records 
are often identified using the Social Security number of the customer as the key. A hashing 
function h assigns memory location h(k) to the record that has k as its key. 
In practice, many different hashing functions are used. One of the most common is the 
function 


h(k) = k mod m 


where m is the number of available memory locations. 

Hashing functions should be easily evaluated so that files can be quickly located. The 
hashing function h(k) = k mod m meets this requirement; to find h(k), we need only compute 
the remainder when k is divided by m. Furthermore, the hashing function should be onto, so that 
all memory locations are possible. The function h(k) = k mod m also satisfies this property. 
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EXAMPLE 1 


EXAMPLE 2 


Links 


Find the memory locations assigned by the hashing function h(k) = k mod 111 to the records 
of customers with Social Security numbers 064212848 and 037149212. 


Solution: The record of the customer with Social Security number 064212848 is assigned to 
memory location 14, because 


h(064212848) = 064212848 mod 111 = 14. 
Similarly, because 
h(037149212) = 037149212 mod 111 = 65, 


the record of the customer with Social Security number 037149212 is assigned to memory 
location 65. < 


Because a hashing function is not one-to-one (because there are more possible keys than 
memory locations), more than one file may be assigned to a memory location. When this happens, 
we say that a collision occurs. One way to resolve a collision is to assign the first free location 
following the occupied memory location assigned by the hashing function. 


After making the assignments of records to memory locations in Example 1, assign a memory 
location to the record of the customer with Social Security number 107405723. 


Solution: First note that the hashing function h(k) = k mod 111 maps the Social Security 
number 107405723 to location 14, because 


h(107405723) = 107405723 mod 111 = 14. 


However, this location is already occupied (by the file of the customer with Social Security 
number 064212848). But, because memory location 15, the first location following memory 
location 14, is free, we assign the record of the customer with Social Security number 107405723 
to this location. < 


In Example | we used a linear probing function, namely h(k,i) = h(k) +i mod m, to 
look for the first free memory location, where i runs from 0 to m — 1. There are many other 
ways to resolve collisions that are discussed in the references on hashing functions given at the 
end of the book. 


Pseudorandom Numbers 


Randomly chosen numbers are often needed for computer simulations. Different methods have 
been devised for generating numbers that have properties of randomly chosen numbers. Because 
numbers generated by systematic methods are not truly random, they are called pseudorandom 
numbers. 

The most commonly used procedure for generating pseudorandom numbers is the 
linear congruential method. We choose four integers: the modulus m, multiplier a, 
increment c, and seed xo, with 2 <a <m,0<c <™m, and 0 < xo < m. We generate a se- 
quence of pseudorandom numbers {x,}, with 0 < x, < m for all n, by successively using the 
recursively defined function 


Xn4+1 = (axXn +c) mod m. 


(This is an example of a recursive definition, discussed in Section 5.3. In that section we will 
show that such sequences are well defined.) 


EXAMPLE 3 
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Many computer experiments require the generation of pseudorandom numbers between 0 
and 1. To generate such numbers, we divide numbers generated with a linear congruential 
generator by the modulus: that is, we use the numbers x,,/m. 


Find the sequence of pseudorandom numbers generated by the linear congruential method with 
modulus m = 9, multiplier a = 7, increment c = 4, and seed xo = 3. 


Solution: We compute the terms of this sequence by successively using the recursively defined 
function x41 = (7x, + 4) mod 9, beginning by inserting the seed x9 = 3 to find x;. We find 
that 


x; = 7x9 + 4mod 9 = 7-3+4mod 9 = 25 mod 9 = 7, 
x2 = 7x; + 4mod 9 = 7-7+4 mod 9 = 53 mod 9 = 8, 
x3 = 7xo +4 mod 9=7-8-+4 mod 9 = 60 mod 9 = 6, 
x4 = 7x3 +4mod 9 = 7-6+4mod 9 = 46 mod 9 = 1, 
x5 = 7x4 +4 mod 9=7-1+4mod 9 = 11 mod 9 = 2, 
x6 = 7x5 +4mod 9=7-2+4mod 9 = 18 mod 9 = 0, 
x7 = 7x6 +4mod9=7-0+4mod9= 4mod9 = 4, 
xg = 7x7 +4mod9=7-4+4 mod 9 = 32 mod 9 = 5, 
x9 = 7xg +4 mod 9 = 7-5+4 mod 9 = 39 mod 9 = 3. 


Because x9 = x9 and because each term depends only on the previous term, we see that the 
sequence 


3,7, 8,6, 1,2,0,4,5, 3,7, 8,6, 1,2,0,4,5,3,... 
is generated. This sequence contains nine different numbers before repeating. < 


Most computers do use linear congruential generators to generate pseudorandom numbers. 
Often, a linear congruential generator with increment c = 0 is used. Such a generator is called 
a pure multiplicative generator. For example, the pure multiplicative generator with modulus 
2?! — 1 and multiplier 7> = 16,807 is widely used. With these values, it can be shown that 
23! _ 2 numbers are generated before repetition begins. 

Pseudorandom numbers generated by linear congruential generators have long been used 
for many tasks. Unfortunately, it has been shown that sequences of pseudorandom numbers gen- 
erated in this way do not share some important statistical properties that true random numbers 
have. Because of this, it is not advisable to use them for some tasks, such as large simulations. 
For such sensitive tasks, other methods are used to produce sequences of pseudorandom num- 
bers, either using some sort of algorithm or sampling numbers arising from a random physical 
phenomenon. For more details on pseudorandom number, see [Kn97] and [Re10]. 


Check Digits 


Congruences are used to check for errors in digit strings. A common technique for detecting 
errors in such strings is to add an extra digit at the end of the string. This final digit, or check digit, 
is calculated using a particular function. Then, to determine whether a digit string is correct, a 
check is made to see whether this final digit has the correct value. We begin with an application 
of this idea for checking the correctness of bit strings. 
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EXAMPLE 4 


EXAMPLE 5 


Parity Check Bits Digital information is represented by bit string, split into blocks of a 
specified size. Before each block is stored or transmitted, an extra bit, called a parity check bit, 
can be appended to each block. The parity check bit x,,, for the bit string x)x2 ...x, is defined 
by 


Xntp = Xp +x. +--+ +x, mod 2. 


It follows that x,+1 is 0 if there are an even number of | bits in the block of n bits and it is 1 if 
there are an odd number of | bits in the block of n bits. When we examine a string that includes 
a parity check bit, we know that there is an error in it if the parity check bit is wrong. However, 
when the parity check bit is correct, there still may be an error. A parity check can detect an odd 
number of errors in the previous bits, but not an even number of errors. (See Exercise 14.) 

Suppose we receive in a transmission the bit strings 01100101 and 11010110, each ending 
with a parity check bit. Should we accept these bit strings as correct? 


Solution: Before accepting these strings as correct, we examine their parity check bits. The 
parity check bit of the first string is 1. Because 0+ 1+1+0+0+1+0 = 1 (mod 2), the 
parity check bit is correct. The parity check bit of the second string is 0. We find that 1 + 1 + 
0+1+0+1+1 = 1 (mod 2), so the parity check is incorrect. We conclude that the first 
string may have been transmitted correctly and we know for certain that the second string was 
transmitted incorrectly. We accept the first string as correct (even though it still may contain an 
even number of errors), but we reject the second string. < 


Check bits computed using congruences are used extensively to verify the correctness of 
various kinds of identification numbers. Examples 5 and 6 show how check bits are computed 
for codes that identify products (Universal Product Codes) and books (International Standard 
Book Numbers). The preambles to Exercises 18, 28, and 32 introduce the use of congruences 
to find and use check digits in money order numbers, airline ticket numbers, and identification 
numbers for periodicals, respectively. Note that congruences are also used to compute check 
digits for bank account numbers, drivers license numbers, credit card numbers, and many other 
types of identification numbers. 


UPCs_ Retail products are identified by their Universal Product Codes (UPCs). The most 
common form of a UPC has 12 decimal digits: the first digit identifies the product category, the 
next five digits identify the manufacturer, the following five identify the particular product, and 
the last digit is a check digit. The check digit is determined by the congruence 


3x, + x2 + 3x3 +.x4 4+ 3x5 +.x6 + 3x7 +xg + 3X9 + X19 + 3X11 + X12 = 0 (mod 10). 


Answer these questions: 


(a) Suppose that the first 11 digits of a UPC are 79357343104. What is the check digit? 
(b) Is 041331021641 a valid UPC? 


Solution: (a) We insert the digits of 79357343104 into the congruence for UPC 
check digits. This gives 3-7+9+3-34+5+3-74+3+4+3-44+3+4+3-14+0+3-4+ 
x12 = 0(mod 10). Simplifying, we have 21+9+9+4+5+4+214+3+4+124+3+4+3+0+412+ 
x12 = 0(mod 10). Hence, 98 + xj2 =0(mod 10). It follows that xj2 = 2 (mod 10), so the 
check digit is 2. 


(b) To check whether 041331021641 is valid, we insert the digits into the congruence these digits 
must satisfy. This gives 3-0+4+3-14+3+4+3-34+14+3-04+24+3-14+6+3-44+1= 
04+4434+34+9414+042+4+3+4+6+ 124+ 1=4#0(mod 10). Hence, 041331021641 
is not a valid UPC. < 
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EXAMPLE 6  ISBNs_ All books are identified by an International Standard Book Number (ISBN-10), a 


Remember that the check 
digit of an ISBN-10 can 
be an X! 


Publishers sometimes do 
not calculate ISBNs 
correctly for their books, 
as was done for an earlier 
edition of this text. 


10-digit code x1x2 ... x10, assigned by the publisher. (Recently, a 13-digit code known as ISBN- 
13 was introduced to identify a larger number of published works; see the preamble to Exercise 
42 in the Supplementary Exercises.) An ISBN-10 consists of blocks identifying the language, 
the publisher, the number assigned to the book by its publishing company, and finally, a check 
digit that is either a digit or the letter X (used to represent 10). This check digit is selected so 
that 


9 
x10 = Sie (mod 11), 


i=1 
or equivalently, so that 


10 


pa = 0 (mod 11). 


i=1 


Answer these questions about ISBN- 10s: 


(a) The first nine digits of the ISBN-10 of the sixth edition of this book are 007288008. What is 
the check digit? 


(b) Is 084930149X a valid ISBN-10? 


Solution: (a) The check digit is determined by the congruence ~ ix; = 0(mod 11). Inserting 
the digits 007288008 gives x19 = 1-0+2-0+3-7+4-24+5-8+6:-84+7-0+8-0+ 
9-8 (mod 11). This means that x39 =0+0+21+8+440+4+ 48+ 0+0+4 72 (mod 11), so 
x19 = 189 = 2 (mod 11). Hence, x19 = 2. 


(b) To see whether 084930149X is a valid ISBN-10, we see if }7}°, ix; = 0 (mod 11). We 
see that 1:04+2.84 3-444.945-346-047+148-449.9+4 10-10=0416+4 
12+ 364+ 15+04+7+4 32+ 814 100 = 299 = 2 4 0 (mod 11). Hence, 084930149X is not 
a valid ISBN-10. < 


Several kinds of errors often arise in identification numbers. A single error, an error in one 
digit of an identification number, is perhaps the most common type of error. Another common 
kind of error is a transposition error, which occurs when two digits are accidentally inter- 
changed. For each type of identification number, including a check digit, we would like to be 
able to detect these common types of errors, as well as other types of errors. We will investigate 
whether the check digit for ISBNs can detect single errors and transposition errors. Whether 
check digits for UPCs can detect these kinds of errors is left as Exercises 26 and 27. 

Suppose that x;x2 ...x 9 is a valid ISBN (so that bara x; = 0 (mod 10)). We will show that 
we can detect a single error and a transposition of two digits (where we include the possibility 
that one of the two digits is the check digit X, representing 10). Suppose that this ISBN has been 
printed with a single error as y; yo... y,0. If there is a single error, then, for some integer /j, 
yi = x; fori A j and yj; = x; +a where —10 <a < 10 anda ¥ 0. Note that a = y; — x; is 
the error in the jth place. It then follows that 


10 10 


\- ivi = (So ixi) + ja = ja £0 (mod 11). 


i=1 i=] 
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These last two congruences hold because er x; =0(mod 10) and 11 / ja, because 11 J j 
and 11 / a. We conclude that y; yz... yj9 is not a valid ISBN. So, we have detected the single 


error. 


Now suppose that two unequal digits have been transposed. It follows that there are distinct 
integers j and k such that y; = x, and yy = x;, and yj = x; fori A j andi #k. Hence, 


10 


10 
> ivi = (> ix) + (jx~ — fxj) + (kx; — kxg) = Gi — k) (xu — xj) 40 (mod 11), 


i=1 i=1 


because ee x; =0(mod 10) and 11 ¥ (j —k) and 11 4 (xy — xj). We see that y;y2... y10 
is not a valid ISBN. Thus, we can detect the interchange of two unequal digits. 


Exercises 


1. Which memory locations are assigned by the hashing 
function h(k) = k mod 97 to the records of insurance 
company customers with these Social Security numbers? 


a) 034567981 b) 183211232 
ec) 220195744 d) 987255335 
2. Which memory locations are assigned by the hashing 
function h(k) = k mod 101 to the records of insurance 
company customers with these Social Security numbers? 
a) 104578690 b) 432222187 
c) 372201919 d) 501338753 
3. A parking lot has 31 visitor spaces, numbered from 0 to 
30. Visitors are assigned parking spaces using the hashing 
function h(k) = k mod 31, where k is the number formed 
from the first three digits on a visitor’s license plate. 
a) Which spaces are assigned by the hashing function to 
cars that have these first three digits on their license 
plates: 317, 918, 007, 100, 111, 310? 


b) Describe a procedure visitors should follow to find a 
free parking space, when the space they are assigned 
is occupied. 


Another way to resolve collisions in hashing is to use double 
hashing. We use an initial hashing function h(k) = k mod p 
where p is prime. We also use a second hashing function 
g(k) = (k + 1) mod (p — 2). When acollision occurs, we use 
a probing sequence h(k, i) = (h(k) +i - g(k)) mod p. 


4. Use the double hashing procedure we have described with 
p = 4969 to assign memory locations to files for em- 
ployees with social security numbers kj = 132489971, 
ky = 509496993, k3 = 546332190, k4 = 034367980, 
ks = 047900151, ke = 329938157, k7 = 212228844, 
kg = 325510778, kg = 353354519, kig = 053708912. 


5. What sequence of pseudorandom numbers is gener- 
ated using the linear congruential generator x,41 = 
(3x, + 2) mod 13 with seed x9 = 1? 


6. What sequence of pseudorandom numbers is gener- 
ated using the linear congruential generator x,41 = 
(4x, + 1) mod 7 with seed x9 = 3? 


7. What sequence of pseudorandom numbers is gener- 
ated using the pure multiplicative generator x41 = 
3x, mod 11 with seed x9 = 2? 

8. Write an algorithm in pseudocode for generating a se- 
quence of pseudorandom numbers using a linear congru- 
ential generator. 

The middle-square method for generating pseudorandom 
numbers begins with an n-digit integer. This number is 
squared, initial zeros are appended to ensure that the result 
has 2n digits, and its middle n digits are used to form the next 
number in the sequence. This process is repeated to generate 
additional terms. 

9. Find the first eight terms of the sequence of four-digit 
pseudorandom numbers generated by the middle square 
method starting with 2357. 

10. Explain why both 3792 and 2916 would be bad choices 
for the initial term of a sequence of four-digit pseudoran- 
dom numbers generated by the middle square method. 

The power generator is a method for generating pseudoran- 

dom numbers. To use the power generator, parameters p and d 

are specified, where p is a prime, d is a positive integer such 

that p / d, and aseed xo is specified. The pseudorandom num- 
bers x1, X2,... are generated using the recursive definition 

Xn41 = ae mod p. 

11. Find the sequence of pseudorandom numbers generated 
by the power generator with p= 7, d = 3, and seed 
xo = oh 

12. Find the sequence of pseudorandom numbers generated 
by the power generator with p = 11, d = 2, and seed 
xo = 3. 

13. Suppose you received these bit strings over a communi- 
cations link, where the last bit is a parity check bit. In 
which string are you sure there is an error? 

a) 00000111111 
b) 10101010101 
ce) 11111100000 
d) 10111101111 

14. Prove that a parity check bit can detect an error in a string 

if and only if the string contains an odd number of errors. 


15. The first nine digits of the ISBN-10 of the European ver- 
sion of the fifth edition of this book are 0-07-119881. 
What is the check digit for that book? 

16. The ISBN-10 of the sixth edition of Elementary Number 
Theory and Its Applications is 0-321-500Q1-8, where Q 
is a digit. Find the value of Q. 

17. Determine whether the check digit of the ISBN-10 for 
this textbook (the seventh edition of Discrete Mathemat- 
ics and its Applications) was computed correctly by the 
publisher. 

The United States Postal Service (USPS) sells money orders 

identified by an 11-digit number x1 x2 . . . x11. The first ten dig- 

its identify the money order; x1; is a check digit that satisfies 

Xyy =X, +x2+---+x19 mod 9. 

18. Find the check digit for the USPS money orders that have 
identification number that start with these ten digits. 

a) 7555618873 
b) 6966133421 
c) 8018927435 
d) 3289744134 

19. Determine whether each of these numbers is a valid USPS 
money order identification number. 
a) 74051489623 
b) 88382013445 
c) 56152240784 
d) 66606631178 

20. One digit in each of these identification numbers of a 
postal money order is smudged. Can you recover the 
smudged digit, indicated by a Q, in each of these num- 
bers? 


a) Q1223139784 
b) 67021200988 
c) 27041007734 
d) 213279032Q1 

21. One digit in each of these identification numbers of a 
postal money order is smudged. Can you recover the 
smudged digit, indicated by a Q, in each of these num- 
bers? 
a) 49321200688 
b) 85009103858 
c) 20941007734 
d) 66687003201 

22. Determine which single digit errors are detected by the 
USPS money order code. 

23. Determine which transposition errors are detected by the 
USPS money order code. 

24. Determine the check digit for the UPCs that have these 
initial 11 digits. 
a) 73232184434 
b) 63623991346 
c) 04587320720 
d) 93764323341 

25. Determine whether each of the strings of 12 digits is a 
valid UPC code. 
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a) 036000291452 
b) 012345678903 
c) 782421843014 
d) 726412175425 
26. Does the check digit of a UPC code detect all single er- 
rors? Prove your answer or find a counterexample. 
27. Determine which transposition errors the check digit of 
a UPC code finds. 
Some airline tickets have a 15-digit identification number 
a\az...a 15 where aj5 is acheck digit that equals aja2 ...ay4 
mod 7. 
28. Find the check digit a;5 that follows each of these initial 
14 digits of an airline ticket identification number. 
a) 10237424413392 
b) 00032781811234 
c) 00611232134231 
d) 00193222543435 
29. Determine whether each of these 15-digit numbers is a 
valid airline ticket identification number. 
a) 101333341789013 
b) 007862342770445 
c) 113273438882531 
d) 000122347322871 
30. Which errors in a single digit of a 15-digit airline ticket 
identification number can be detected? 

* 31. Can the accidental transposition of two consecutive dig- 
its in an airline ticket identification number be detected 
using the check digit? 

Periodicals are identified using an International Standard 
Serial Number (ISSN). An ISSN consists of two blocks 
of four digits. The last digit in the second block is a check 
digit. This check digit is determined by the congruence dg = 
3d, + 4dz + 5d3 + 6d4 + 7ds5 + 8d6 + 9d7 (mod 11). When 
dg = 10 (mod 11), we use the letter X to represent dg in the 
code. 
32. For each of these initial seven digits of an ISSN, deter- 
mine the check digit (which may be the letter X). 
a) 1570-868 
b) 1553-734 
c) 1089-708 
d) 1383-811 
33. Are each of these eight-digit codes possible ISSNs? That 
is, do they end with a correct check digit? 
a) 1059-1027 
b) 0002-9890 
c) 1530-8669 
d) 1007-120X 
34. Does the check digit of an ISSN detect every single error 
in an ISSN? Justify your answer with either a proof or a 
counterexample. 
35. Does the check digit of an ISSN detect every error where 
two consecutive digits are accidentally interchanged? Jus- 
tify your answer with either a proof or a counterexample. 
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Introduction 


Number theory plays a key role in cryptography, the subject of transforming information so that 
it cannot be easily recovered without special knowledge. Number theory is the basis of many 
classical ciphers, first used thousands of years ago, and used extensively until the 20th century. 
These ciphers encrypt messages by changing each letter to a different letter, or each block of 
letters to a different block of letters. We will discuss some classical ciphers, including shift 
ciphers, which replace each letter by the letter a fixed number of positions later in the alphabet, 
wrapping around to the beginning of the alphabet when necessary. The classical ciphers we will 
discuss are examples of private key ciphers where knowing how to encrypt allows someone to 
also decrypt messages. With a private key cipher, two parties who wish to communicate in secret 
must share a secret key. The classical ciphers we will discuss are also vulnerable to cryptanalysis, 
which seeks to recover encrypted information without access to the secret information used to 
encrypt the message. We will show how to cryptanalyze messages sent using shift ciphers. 

Number theory 1s also important in public key cryptography, a type of cryptography invented 
in the 1970s. In public key cryptography, knowing how to encrypt does not also tell someone 
how to decrypt. The most widely used public key system, called the RSA cryptosystem, encrypts 
messages using modular exponentiation, where the modulus is the product of two large primes. 
Knowing how to encrypt requires that someone know the modulus and an exponent. (It does 
not require that the two prime factors of the modulus be known.) As far as it is known, knowing 
how to decrypt requires someone to know how to invert the encryption function, which can only 
be done in a practical amount of time when someone knows these two large prime factors. In 
this chapter we will explain how the RSA cryptosystem works, including how to encrypt and 
decrypt messages. 

The subject of cryptography also includes the subject of cryptographic protocols, which are 
exchanges of messages carried out by two or more parties to achieve a specific security goal. We 
will discuss two important protocols in this chapter. One allows two people to share a common 
secret key. The other can be used to send signed messages so that a recipient can be sure that 
they were sent by the purported sender. 


Classical Cryptography 


One of the earliest known uses of cryptography was by Julius Caesar. He made messages secret 
by shifting each letter three letters forward in the alphabet (sending the last three letters of the 
alphabet to the first three). For instance, using this scheme the letter B is sent to E and the letter 
X is sent to A. This is an example of encryption, that is, the process of making a message secret. 

To express Caesar’s encryption process mathematically, first replace each letter by an ele- 
ment of Zy6, that is, an integer from 0 to 25 equal to one less than its position in the alphabet. For 
example, replace A by 0, K by 10, and Z by 25. Caesar’s encryption method can be represented 
by the function f that assigns to the nonnegative integer p, p < 25, the integer f(p) in the set 
{0, 1, 2,..., 25} with 


f (p) = (p + 3) mod 26. 


In the encrypted version of the message, the letter represented by p is replaced with the letter 
represented by (p + 3) mod 26. 
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EXAMPLE1_ What is the secret message produced from the message “MEET YOU IN THE PARK” using 


EXAMPLE 2 


EXAMPLE 3 


the Caesar cipher? 

Solution: First replace the letters in the message with numbers. This produces 
124419 24 14 20 8 13 1974 150 17 10. 

Now replace each of these numbers p by f(p) = (p + 3) mod 26. This gives 
157722 117 23 11 16 22 107 18 3 20 13. 


Translating this back to letters produces the encrypted message “PHHW BRX LQ WKH 
SDUN.” < 


To recover the original message from a secret message encrypted by the Caesar cipher, the 
function f—!, the inverse of f, is used. Note that the function f—! sends an integer p from 
Z6, to f—'(p) = (p — 3) mod 26. In other words, to find the original message, each letter is 
shifted back three letters in the alphabet, with the first three letters sent to the last three letters 
of the alphabet. The process of determining the original message from the encrypted message 
is called decryption. 

There are various ways to generalize the Caesar cipher. For example, instead of shifting the 
numerical equivalent of each letter by 3, we can shift the numerical equivalent of each letter by 
k, so that 


f(p) = (p +k) mod 26. 
Such a cipher is called a shift cipher. Note that decryption can be carried out using 
f-'(p) = (p —k) mod 26. 


Here the integer k is called a key. We illustrate the use of a shift cipher in Examples 2 and 3. 


Encrypt the plaintext message “STOP GLOBAL WARMING?” using the shift cipher with shift 
k=11. 


Solution: To encrypt the message “STOP GLOBAL WARMING? we first translate each letter 
to the corresponding element of Z2¢. This produces the string 


18 19 14 15 611141011 22017128136. 
We now apply the shift f(p) = (p + 11) mod 26 to each number in this string. We obtain 
34250 17 22 25 12 11 22 711223 19 24 17. 


Translating this last string back to letters, we obtain the ciphertext “DEZA RWZMLW HLCX- 
TYR.” < 
Decrypt the ciphertext message “LEWLYPLUJL PZ H NYLHA ALHJOLY” that was en- 
crypted with the shift cipher with shift k = 7. 


Solution: To decrypt the ciphertext “LEWLYPLUJL PZ H NYLHA ALHJOLY” we first 
translate the letters back to elements of Zo¢. We obtain 


11422 11 24151120911 15 25 7 13 241170 01179 14 11 24. 
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EXAMPLE 4 


Mathematicians make the 
best code breakers. Their 
work in World War II 
changed the course of the 
war. 


Next, we shift each of these numbers by —k = —7 modulo 26 to obtain 
423 15417841324 8 18 0 6174019 194027417. 


Finally, we translate these numbers back to letters to obtain the plaintext. We obtain 
“EXPERIENCE IS A GREAT TEACHER.” 


We can generalize shift ciphers further to slightly enhance security by using a function of 
the form 


f(p) = (ap + b) mod 26, 


where a and Db are integers, chosen so that f is a bijection. (The function f(p) = (ap + 
b) mod 26 is a bijection if and only if gcd(a, 26) = 1.) Such a mapping is called an affine 
transformation, and the resulting cipher is called an affine cipher. 


What letter replaces the letter K when the function f(p) = (7p + 3) mod 26 is used for en- 
cryption? 


Solution: First, note that 10 represents K. Then, using the encryption function specified, it 
follows that f(10) = (7-10 +3) mod 26 = 21. Because 21 represents V, K is replaced by V 
in the encrypted message. < 


We will now show how to decrypt messages encrypted using an affine cipher. Suppose that 
c = (ap + b) mod 26 with gcd(a, 26) = 1. To decrypt we need to show how to express p in 
terms of c. To do this, we apply the encrypting congruence c = ap + b(mod 26), and solve it 
for p. To do this, we first subtract b from both sides, to obtain c — b = ap (mod 26). Because 
gcd(a, 26) = 1, we know that there is an inverse a of a modulo 26. Multiplying both sides of 
the last equation by a gives us a(c — b) = aap (mod 26). Because aa = 1 (mod 26), this tells 
us that p = a(c — b) (mod 26). This determines p because p belongs to Z26. 


CRYPTANALYSIS The process of recovering plaintext from ciphertext without knowledge 
of both the encryption method and the key is known as crytanalysis or breaking codes. In 
general, cryptanalysis is a difficult process, especially when the encryption method is unknown. 
We will not discuss cryptanalysis in general, but we will explain how to break messages that 
were encrypted using a shift cipher. 

If we know that a ciphertext message was produced by enciphering a message using a shift 
cipher, we can try to recover the message by shifting all characters of the ciphertext by each 
of the 26 possible shifts (including a shift of zero characters). One of these is guaranteed to be 
the plaintext. However, we can use a more intelligent approach, which we can build upon to 
cryptanalyze ciphertext resulting from other ciphers. The main tool for cryptanalyzing ciphertext 
encrypted using a shift cipher is the count of the frequency of letters in the ciphertext. The nine 
most common letters in English text and their approximate relative frequencies are E 13%, T 9%, 
A 8%, 0 8%, 17%, N 7%, S 7%, H 6%, and R 6%. To cryptanalyze ciphertext that we know was 
produced using a shift cipher, we first find the relative frequencies of letters in the ciphertext. 
We list the most common letters in the ciphertext in frequency order; we hypothesize that the 
most common letter in the ciphertext is produced by encrypting E. Then, we determine the 
value of the shift under this hypothesis, say k. If the message produced by shifting the ciphertext 
by —k makes sense, we presume that our hypothesis is correct and that we have the correct 
value of k. If it does not make sense, we next consider the hypothesis that the most common 
letter in the ciphertext is produced by encrypting T, the second most common letter in English; 
we find k under this hypothesis, shift the letters of the message by —k, and see whether the 
resulting message makes sense. If it does not, we continue the process working our way through 
the letters from most common to least common. 


EXAMPLE 5 


Links 


EXAMPLE 6 


DEFINITION 1 
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Suppose that we intercepted the ciphertext message ZNK KGXRE HOXJ MKZY ZNK CUXS 
that we know was produced by a shift cipher. What was the original plaintext message? 


Solution: Because we know that the intercepted ciphertext message was encrypted using a 
shift cipher, we begin by calculating the frequency of letters in the ciphertext. We find that 
the most common letter in the ciphertext is K. So, we hypothesize that the shift cipher sent 
the plaintext letter E to the ciphertext letter K. If this hypothesis is correct, we know that 
10=4+k mod 26, so k = 6. Next, we shift the letters of the message by —6, obtaining 
THE EARLY BIRD GETS THE WORM. Because this message makes sense, we assume that 
the hypothesis that k = 6 is correct. < 


BLOCK CIPHERS Shift ciphers and affine ciphers proceed by replacing each letter of the 
alphabet by another letter in the alphabet. Because of this, these ciphers are called character 
or monoalphabetic ciphers. Encryption methods of this kind are vulnerable to attacks based 
on the analysis of letter frequency in the ciphertext, as we just illustrated. We can make it 
harder to successfully attack ciphertext by replacing blocks of letters with other blocks of letters 
instead of replacing individual characters with individual characters; such ciphers are called 
block ciphers. 

We will now introduce a simple type of block cipher, called the transposition cipher. As 
a key we use a permutation o of the set {1,2,...,m} for some positive integer m, that is, a 
one-to-one function from {1,2,...,m} to itself. To encrypt a message we first split its letters 
into blocks of size m. (If the number of letters in the message is not divisible by m we add 
some random letters at the end to fill out the final block.) We encrypt the block pj p2... pm 
as C1C2...Cm = Po(1)Po(2) «+++ Po(m). To decryt a ciphertext block c)c2...cm, we transpose 
its letters using the permutation o~!, the inverse of o. Example 6 illustrates encryption and 
decryption for a transposition cipher. 


Using the transposition cipher based on the permutation o of the set {1, 2, 3, 4} with o(1) = 3, 
o(2) = 1,0(3) = 4, and o (4) = 2, 


(a) Encrypt the plaintext message PIRATE ATTACK. 


(b) Decrypt the ciphertext message SWUE TRAE OEHS, which was encrypted using this cipher. 


Solution: (a) We first split the letters of the plaintext into blocks of four letters. We obtain PIRA 
TEAT TACK. To encrypt each block, we send the first letter to the third position, the second 
letter to the first position, the third letter to the fourth position, and the fourth letter to the second 
position. We obtain IAPR ETTA AKTC. 

(b) We note that ae the inverse of o, sends 1 to 2, sends 2 to 4, sends 3 to 1, and sends 
4 to 3. Applying o~!(m) to each block gives us the plaintext: USEW ATER HOSE. (Grouping 
together these letters to form common words, we surmise that the plaintext is USE WATER 
HOSE.) < 


CRYPTOSYSTEMS We have defined two families of ciphers: shift ciphers and affine ciphers. 
We now introduce the notion of a cryptosystem, which provides a general structure for defining 
new families of ciphers. 


A cryptosystem is a five-tuple (P,C, K,€, D), where P is the set of plaintext strings, C is 
the set of ciphertext strings, K is the keyspace (the set of all possible keys), € is the set of 
encryption functions, and D is the set of decryption functions. We denote by E; the encryption 
function in € corresponding to the key k and D, the decryption function in D that decrypts 
ciphertext that was encrypted using Ex, that is D,(Ex(p)) = p, for all plaintext strings p. 
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We now illustrate the use of the definition of a cryptosystem. 


EXAMPLE 7 _ Describe the family of shift ciphers as a cryptosytem. 


Solution: To encrypt a string of English letters with a shift cipher, we first translate each letter 
to an integer between 0 and 25, that is, to an element of Zo¢. We then shift each of these integers 
by a fixed integer modulo 26, and finally, we translate the integers back to letters. To apply the 
defintion of a cryptosystem to shift ciphers, we assume that our messages are already integers, 
that is, elements of Zo¢. That is, we assume that the translation between letters and integers 
is outside of the cryptosystem. Consequently, both the set of plaintext strings P and the set of 
ciphertext strings C are the set of strings of elements of Z2¢. The set of keys K is the set of possible 
shifts, so K = Za6. The set € consists of functions of the form Ex(p) = (p + k) mod 26, and 
the set D of decryption functions is the same as the set of encrypting functions where D;(p) = 
(p — k) mod 26. 


The concept of a cryptosystem is useful in the discussion of additional families of ciphers 
and is used extensively in cryptography. 


Public Key Cryptography 


All classical ciphers, including shift ciphers and affine ciphers, are examples of private key 
cryptosystems. In a private key cryptosystem, once you know an encryption key, you can 
quickly find the decryption key. So, knowing how to encrypt messages using a particular key 
allows you to decrypt messages that were encrypted using this key. For example, when a shift 
cipher is used with encryption key k, the plaintext integer p is sent to 


c= (p+k) mod 26. 
Decryption is carried out by shifting by —k; that is, 
p = (c —k) mod 26. 


So knowing how to encrypt with a shift cipher also tells you how to decrypt. 

When a private key cryptosystem is used, two parties who wish to communicate in secret 
must share a secret key. Because anyone who knows this key can both encrypt and decrypt 
messages, two people who want to communicate securely need to securely exchange this key. 
(We will introduce a method for doing this later in this section.) The shift cipher and affine cipher 
cryptosystems are private key cryptosystems. They are quite simple and are extremely vulnerable 
to cryptanalysis. However, the same is not true of many modern private key cryptosystems. In 
particular, the current US government standard for private key cryptography, the Advanced 
Encryption Standard (AES), is extremely complex and is considered to be highly resistant to 
cryptanalysis. (See [St06] for details on AES and other modern private key cryptosystems.) 
AES is widely used in government and commercial communications. However, it still shares 
the property that for secure communications keys be shared. Furthermore, for extra security, a 
new key is used for each communication session between two parties, which requires a method 
for generating keys and securely sharing them. 

To avoid the need for keys to be shared by every pair of parties that wish to communicate 
securely, in the 1970s cryptologists introduced the concept of public key cryptosystems. When 
such cryptosystems are used, knowing how to send an encrypted message does not help decrypt 
messages. In such a system, everyone can have a publicly known encryption key. Only the 
decryption keys are kept secret, and only the intended recipient of a message can decrypt it, 
because, as far as it is currently known, knowledge of the encryption key does not let someone 
recover the plaintext message without an extraordinary amount of work (such as billions of 
years of computer time). 


M.LT. is also known as 
the ’Tute. 


Unfortunately, no one 
calls this the Cocks 
cryptosystem. 
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The RSA Cryptosystem 


In 1976, three researchers at the Massachusetts Institute of Technology—Ronald Rivest, Adi 
Shamir, and Leonard Adleman—introduced to the world a public key cryptosystem, known as the 
RSA system, from the initials of its inventors. As often happens with cryptographic discoveries, 
the RSA system had been discovered several years earlier in secret government research in the 
United Kingdom. Clifford Cocks, working in secrecy at the United Kingdom’s Government 
Communications Headquarters (GCHQ), had discovered this cryptosystem in 1973. However, 
his invention was unknown to the outside world until the late 1990s, when he was allowed to 
share classified GCHQ documents from the early 1970s. (An excellent account of this earlier 
discovery, as well as the work of Rivest, Shamir, and Adleman, can be found in [Si99].) 

In the RSA cryptosystem, each individual has an encryption key (n, e) where n = pq, the 
modulus is the product of two large primes p and qg, say with 200 digits each, and an exponent 
e that is relatively prime to (p — 1)(q — 1). To produce a usable key, two large primes must be 
found. This can be done quickly on a computer using probabilistic primality tests, referred to 
earlier in this section. However, the product of these primes n = pq, with approximately 400 
digits, cannot, as far as is currently known, be factored in a reasonable length of time. As we 
will see, this is an important reason why decryption cannot, as far as is currently known, be 
done quickly without a separate decryption key. 


RSA Encryption 


To encrypt messages using a particular key (n, e), we first translate a plaintext message M 
into sequences of integers. To do this, we first translate each plaintext letter into a two-digit 
number, using the same translation we employed for shift ciphers, with one key difference. 
That is, we include an initial zero for the letters A through J, so that A is translated into 00, 
B into O1,..., and J into 09. Then, we concatenate these two-digit numbers into strings of digits. 
Next, we divide this string into equally sized blocks of 2N digits, where 2N is the largest even 
number such that the number 2525 ...25 with 2N digits does not exceed n. (When necessary, 
we pad the plaintext message with dummy Xs to make the last block the same size as all other 
blocks.) 

After these steps, we have translated the plaintext message M into a sequence of integers 
m,,™2,..., mg for some integer k. Encryption proceeds by transforming each block mj; to a 
ciphertext block c;. This is done using the function 


C = M* mod n. 


(To perform the encryption, we use an algorithm for fast modular exponentiation, such as 
Algorithm 5 in Section 4.2.) We leave the encrypted message as blocks of numbers and send 
these to the intended recipient. Because the RSA cryptosystem encrypts blocks of characters 
into blocks of characters, it is a block cipher. 


CLIFFORD COCKS (BORN 1950) Clifford Cocks, born in Cheshire, England, was a talented mathematics 
student. In 1968 he won a silver medal at the International Mathematical Olympiad. Cocks attended King’s 
College, Cambridge, studying mathematics. He also spent a short time at Oxford University working in number 
theory. In 1973 he decided not to complete his graduate work, instead taking a mathematical job at the Govern- 
ment Communications Headquarters (GCHQ) of British intelligence. Two months after joining GCHQ, Cocks 
learned about public key cryptography from an internal GCHQ report written by James Ellis. Cocks used his 
number theory knowledge to invent what is now called the RSA cryptosystem. He quickly realized that a public 
key cryptosystem could be based on the difficulty of reversing the process of multiplying two large primes. In 
1997 he was allowed to reveal declassified GCHQ internal documents describing his discovery. Cocks is also 


known for his invention of a secure identity based encryption scheme, which uses information about a user’s identity as a public key. 
In 2001, Cocks became the Chief Mathematician at GCHQ. He has also set up the Heilbronn Institute for Mathematical Research, 
a partnership between GCHQ and the University of Bristol. 


300 4/Number Theory and Cryptography 


EXAMPLE 8 


Example 8 illustrates how RSA encryption is performed. For practical reasons we use small 
primes p and q in this example, rather than primes with 200 or more digits. Although the cipher 
described in this example is not secure, it does illustrate the techniques used in the RSA cipher. 


Encrypt the message STOP using the RSA cryptosystem with key (2537, 13). Note that 2537 = 
43-59, p = 43 and q = 59 are primes, and 


gcd(e, (p — I)(¢g — 1)) = ged(13, 42 - 58) = 1. 


Solution: To encrypt, we first translate the letters in STOP into their numerical equivalents. We 
then group these numbers into blocks of four digits (because 2525 < 2537 < 252525), to obtain 


1819 1415. 
We encrypt each block using the mapping 
C = M*? mod 2537. 


Computations using fast modular multiplication show that 1819!? mod 2537 = 2081 and 
1415!3 mod 2537 = 2182. The encrypted message is 2081 2182. < 


RSA Decryption 


The plaintext message can be quickly recovered from a ciphertext message when the decryp- 
tion key d, an inverse of e modulo (p — 1)(q — 1), is known. [Such an inverse exists because 
gcd(e, (p — 1)(¢ — 1)) = 1.] To see this, note that if de = 1 (mod (p — 1)(q — 1)), there is an 
integer k such that de = 1+ k(p — 1)(q — 1). It follows that 


C4 = (M*)? = M® = M'tk-DG-D (mod n). 


RONALD RIVEST (BORN 1948) Ronald Rivest received a B.A. from Yale in 1969 and his Ph.D. in computer 
science from Stanford in 1974. Rivest is a computer science professor at M.I.T. and was a cofounder of RSA Data 
Security, which held the patent on the RSA cryptosystem that he invented together with Adi Shamir and Leonard 
Adleman. Areas that Rivest has worked in besides cryptography include machine learning, VLSI design, and 
computer algorithms. He is a coauthor of a popular text on algorithms ([CoLeRiSt09]). 


ADI SHAMIR (BORN 1952) Adi Shamir was born in Tel Aviv, Israel. His undergraduate degree is from 
Tel Aviv University (1972) and his Ph.D. is from the Weizmann Institute of Science (1977). Shamir was a 
research assistant at the University of Warwick and an assistant professor at M.I.T. He is currently a profes- 
sor in the Applied Mathematics Department at the Weizmann Institute and leads a group studying computer 
security. Shamir’s contributions to cryptography, besides the RSA cryptosystem, include cracking knapsack 
cryptosystems, cryptanalysis of the Data Encryption Standard (DES), and the design of many cryptographic 
protocols. 


LEONARD ADLEMAN (BORN 1945) Leonard Adleman was born in San Francisco, California. He received 
a B.S. in mathematics (1968) and his Ph.D. in computer science (1976) from the University of California, 
Berkeley. Adleman was a member of the mathematics faculty at M.LT. from 1976 until 1980, where he was 
a coinventor of the RSA cryptosystem, and in 1980 he took a position in the computer science department at 
the University of Southern California (USC). He was appointed to a chaired position at USC in 1985. Adleman 
has worked on computer security, computational complexity, immunology, and molecular biology. He invented 
the term “computer virus.” Adleman’s recent work on DNA computing has sparked great interest. He was a 
technical adviser for the movie Sneakers, in which computer security played an important role. 


EXAMPLE 9 


Links 
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By Fermat’s little theorem [assuming that gcd(M, p) = gcd(M, q) = 1, which holds except in 
rare cases, which we cover in Exercise 28], it follows that M?~! = 1 (mod p) and M@-!= 
1 (mod q). Consequently, 


cl =M.(M?!)kG-) = M.1 = M (mod p) 
and 
C4 =M.(MI-!)k-) = M.1 = M (modg). 
Because gcd(p, q) = 1, it follows by the Chinese remainder theorem that 
C4 = M (mod Pq). 
Example 9 illustrates how to decrypt messages sent using the RSA cryptosystem. 


We receive the encrypted message 0981 0461. What is the decrypted message if it was encrypted 
using the RSA cipher from Example 8? 


Solution: The message was encrypted using the RSA cryptosystem with n = 43 - 59 and expo- 
nent 13. As Exercise 2 in Section 4.4 shows, d = 937 is an inverse of 13 modulo 42 - 58 = 2436. 
We use 937 as our decryption exponent. Consequently, to decrypt a block C, we compute 


M = C?’ mod 2537. 


To decrypt the message, we use the fast modular exponentiation algorithm to compute 
0981937 mod 2537 = 0704 and 04619” mod 2537 = 1115. Consequently, the numerical version 
of the original message is 0704 1115. Translating this back to English letters, we see that the 
message is HELP. < 


RSA as a Public Key System 


Why is the RSA cryptosystem suitable for public key cryptography? First, it is possible to rapidly 
construct a public key by finding two large primes p and q, each with more than 200 digits, 
and to find an integer e relatively prime to (p — 1)(q — 1). When we know the factorization of 
the modulus n, that is, when we know p and q, we can quickly find an inverse d of e modulo 
(p — 1)(q — 1). [This is done by using the Euclidean algorithm to find Bézout coefficients s 
and t for d and (p — 1)(q — 1), which shows that the inverse of d modulo (p — 1)(g — 1) is 
s mod (p — 1)(q — 1).] Knowing d lets us decrypt messages sent using our key. However, no 
method is known to decrypt messages that is not based on finding a factorization of n, or that 
does not also lead to the factorization of n. 

Factorization is believed to be a difficult problem, as opposed to finding large primes p 
and q, which can be done quickly. The most efficient factorization methods known (as of 2010) 
require billions of years to factor 400-digit integers. Consequently, when p and q are 200-digit 
primes, it is believed that messages encrypted using n = pq as the modulus cannot be found in 
a reasonable time unless the primes p and g are known. 

Although no polynomial-time algorithm is known for factoring large integers, active re- 
search is under way to find new ways to efficiently factor integers. Integers that were thought, as 
recently as several years ago, to be far too large to be factored in a reasonable amount of time can 
now be factored routinely. Integers with more than 150 digits, as well as some with more than 
200 digits, have been factored using team efforts. When new factorization techniques are found, 
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it will be necessary to use larger primes to ensure secrecy of messages. Unfortunately, messages 
that were considered secure earlier can be saved and subsequently decrypted by unintended 
recipients when it becomes feasible to factor the n = pq in the key used for RSA encryption. 

The RSA method is now widely used. However, the most commonly used cryptosystems 
are private key cryptosystems. The use of public key cryptography, via the RSA system, is 
growing. Nevertheless, there are applications that use both private key and public key systems. 
For example, a public key cryptosystem, such as RSA, can be used to distribute private keys 
to pairs of individuals when they wish to communicate. These people then use a private key 
system for encryption and decryption of messages. 


Cryptographic Protocols 


So far we have shown how cryptography can be used to make messages secure. However, 
there are many other important applications of cryptography. Among these applications are 
cryptographic protocols, which are exchanges of messages carried out by two or more parties 
to achieve a particular security goal. In particular, we will show how cryptography can be used 
to allow two people to exchange a secret key over an insecure communication channel. We will 
also show how cryptography can be used to send signed secret messages so that the recipient 
can be sure that the message came from the purported sender. We refer the reader to [St05] for 
thorough discussions of a variety of cryptographic protocols. 


KEY EXCHANGE We now discuss a protocol that two parties can use to exchange a secret 
key over an insecure communications channel without having shared any information in the 
past. Generating a key that two parties can share is important for many applications of cryp- 
tography. For example, for two people to send secure messages to each other using a private 
key cryptosystem they need to share a common key. The protocol we will describe is known as 
the Diffie-Hellman key agreement protocol, after Whitfield Diffie and Martin Hellman, who 
described it in 1976. However, this protocol was invented in 1974 by Malcolm Williamson in 
secret work at the British GCHQ. It was not until 1997 that his discovery was made public. 


Suppose that Alice and Bob want to share a common key. The protocol follows these steps, 
where the computations are done in Zp. 


(1) Alice and Bob agree to use a prime p and a primitive root a of p. 
(2) Alice chooses a secret integer k, and sends a‘! mod p to Bob. 
(3) Bob chooses a secret integer kz and sends a mod p to Alice. 


(4) Alice computes (a®2)" mod D. 


hi yka 


(5) Bob computes (a mod p. 


At the end of this protocol, Alice and Bob have computed their shared key, namely 
(a*2)" mod p = (a*')” mod p. 


To analyze the security of this protocol, note that the messages sent in steps (1), (2), and 
(3) are not assumed to be sent securely. We can even assume that these communications were 
in the clear and that their contents are public information. So, p, a, a“! mod p, and a‘? mod p 
are assumed to be public information. The protocol ensures that k, kz, and the common key 
(a’2)" mod p = (a“')” mod p are kept secret. To find the secret information from this pub- 
lic information requires that an adversary solves instances of the discrete logarithm problem, 


EXAMPLE 10 
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because the adversary would need to find k, and ky from a“! mod p and a‘ mod p, respec- 
tively. Furthermore, no other method is known for finding the shared key using just the public 
information. We have remarked that this is thought to be computationally infeasible when p 
and a are sufficiently large. With the computing power available now, this system is considered 
unbreakable when p has more than 300 decimal digits and k, and kz have more than 100 decimal 
digits each. 


DIGITAL SIGNATURES Not only can cryptography be used to secure the confidentiality of 
a message, but it also can be used so that the recipient of the message knows that it came from 
the person they think it came from. We first show how a message can be sent so that a recipient 
of the message will be sure that the message came from the purported sender of the message. 
In particular, we can show how this can be accomplished using the RSA cryptosystem to apply 
a digital signature to a message. 

Suppose that Alice’s RSA public key is (7, e) and her private key is d. Alice encrypts a plain- 
text message x using the encryption function E(,¢)(x) = x© mod n. She decrypts a ciphertext 
message y using the decryption function Din,e) = x4 mod n. Alice wants to send the message 
M so that everyone who receives the message knows that it came from her. Just as in RSA en- 
cryption, she translates the letters into their numerical equivalents and splits the resulting string 
into blocks m,, m2, ..., mx such that each block is the same size which is as large as possible 
so that 0 < m; <n fori = 1,2, ...,k. She then applies her decryption function Diy ,e) to each 
block, obtaining D, -(m;),i = 1, 2,...,k. She sends the result to all intended recipients of the 
message. 

When a recipient receives her message, they apply Alice’s encryption function E(;,¢) to each 
block, which everyone has available because Alice’s key (n, e) is public information. The result 
is the original plaintext block because E(y,¢)(D(n,e)(*)) = x. So, Alice can send her message 
to as many people as she wants and by signing it in this way, every recipient can be sure it came 
from Alice. Example 10 illustrates this protocol. 


Suppose Alice’s public RSA cryptosystem key is the same as in Example 8. That is, n = 
43 - 59 = 2537 and e = 13. Her decryption key is d = 937, as described in Example 9. She 
wants to send the message “MEET AT NOON” to her friends so that they are sure it came from 
her. What should she send? 


Solution: Alice first translates the message into blocks of digits, obtaining 1204 0419 0019 
1314 1413 (as the reader should verify). She then applies her decryption transformation 
D(2537,13) (x) = x°37 mod 2537 to each block. Using fast modular exponentiation (with the 
help of a computational aid), she finds that 120493” mod 2537 = 817, 4199°” mod 2537 = 555, 
19°37 mod 2537 = 1310, 131473” mod 2537 = 2173, and 1413?” mod 2537 = 1026. 

So, the message she sends, split into blocks, is 0817 0555 1310 2173 1026. When one of 
her friends gets this message, they apply her encryption transformation E (2537, 13) to each block. 
When they do this, they obtain the blocks of digits of the original message which they translate 
back to English letters. < 


We have shown that signed messages can be sent using the RSA cryptosystem. We can extend 
this by sending signed secret messages. To do this, the sender applies RSA encryption using 
the publicly known encryption key of an intended recipient to each block that was encrypted 
using sender’s decryption transformation. The recipient then first applies his private decryption 
transformation and then the sender’s public encryption transformation. (Exercise 32 asks for 
this protocol to be carried out.) 
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Exercises 


1. Encrypt the message DO NOT PASS GO by translating 
the letters into numbers, applying the given encryption 
function, and then translating the numbers back into let- 
ters. 


a) f(p) = (p +3) mod 26 (the Caesar cipher) 
b) f(p) = (p + 13) mod 26 
c) f(p) = Gp +7) mod 26 


2. Encrypt the message STOP POLLUTION by translating 
the letters into numbers, applying the given encryption 
function, and then translating the numbers back into let- 
ters. 


a) f(p) = (p +4 mod 26 
b) f(p) = (p +21) mod 26 
ce) f(p) = 7p + 22) mod 26 


3. Encrypt the message WATCH YOUR STEP by translat- 
ing the letters into numbers, applying the given encryp- 
tion function, and then translating the numbers back into 
letters. 


a) f(p) = (p + 14) mod 26 
b) f(p) = (14p + 21) mod 26 
c) f(p) = (-7p + 1) mod 26 


4. Decrypt these messages that were encrypted using the 
Caesar cipher. 


a) EOXH MHDQV 
b) WHVW WRGDB 
c) HDW GLP VXP 


5. Decrypt these messages encrypted using the shift cipher 
f(p) = (p + 10) mod 26. 


a) CEBBOXNOB XYG 
b) LO WI PBSOXN 
c) DSWO PYB PEX 


6. Suppose that when a long string of text is encrypted using 
ashift cipher f(p) = (p + k) mod 26, the most common 
letter in the ciphertext is X. What is the most likely value 
for k assuming that the distribution of letters in the text 
is typical of English text? 


7. Suppose that when a string of English text is encrypted us- 
ing a shift cipher f(p) = (p + k) mod 26, the resulting 
ciphertext is DY CVOOZ ZOBMRKXMO DY NBOKW. 
What was the original plaintext string? 

8. Suppose that the ciphertext DVE CFMV KF NFEUVI, 
REU KYRK ZJ KYV JVVU FW JTZVETV was pro- 
duced by encrypting a plaintext message using a shift 
cipher. What is the original plaintext? 

9. Suppose that the ciphertext ERC WYJJMGMIRXPC 
EHZERGIH XIGLRSPSKC MW MRHMWXM- 
RKYMWLEFPI JVSQ QEKMG was produced by en- 
crypting a plaintext message using a shift cipher. What is 
the original plaintext? 


10. Determine whether there is a key for which the encipher- 
ing function for the shift cipher is the same as the deci- 
phering function. 


11. What is the decryption function for an affine cipher if the 
encryption function is c = (15p + 13) mod 26? 


* 12. Find all pairs of integers keys (a, b) for affine ciphers for 
which the encryption function c = (ap + b) mod 26 is 
the same as the corresponding decryption function. 


13. Suppose that the most common letter and the second 
most common letter in a long ciphertext produced by 
encrypting a plaintext using an affine cipher f(p) = 
(ap + b) mod 26 are Z and J, respectively. What are the 
most likely values of a and b? 


14. Encrypt the message GRIZZLY BEARS using blocks 
of five letters and the transposition cipher based on the 
permutation of {1, 2,3, 4,5} with o(1) = 3, 0(2) =5, 
o(3) = 1,0 (4) = 2, anda (5) = 4. For this exercise, use 
the letter X as many times as necessary to fill out the final 
block of fewer then five letters. 


15. Decrypt the message EABW EFRO ATMR ASIN which 
is the ciphertext produced by encrypting a plaintext mes- 
sage using the transposition cipher with blocks of four 
letters and the permutation o of {1, 2,3, 4} defined by 
o(1) = 3,0(2) = 1,0 (3) = 4, anda (4) = 2. 


*16. Suppose that you know that a ciphertext was produced 
by encrypting a plaintext message with a transposition 
cipher. How might you go about breaking it? 


17. Suppose you have intercepted a ciphertext message and 
when you determine the frequencies of letters in this mes- 
sage, you find the frequencies are similar to the frequency 
of letters in English text. Which type of cipher do you sus- 
pect was used? 


The Vigenére cipher is a block cipher, with a key that is a 
string of letters with numerical equivalents k,k2 ...k»,, where 
ki € Zog for i=1,2,...,m. Suppose that the numerical 
equivalents of the letters of a plaintext block are pj p2... Pm. 
The corresponding numerical ciphertext block is (p; + 
ky) mod 26 (p2 + k2) mod 26... (pm + km) mod 26. Finally, 
we translate back to letters. For example, suppose that the 
key string is RED, with numerical equivalents 17 4 3. 
Then, the plaintext ORANGE, with numerical equivalents 
14 17 00 13 06 04, is encrypted by first splitting it into two 
blocks 14 17 00 and 13 06 04. Then, in each block we shift 
the first letter by 17, the second by 4, and the third by 3. We 
obtain 5 21 03 and 04 10 07. The cipherext is FVDEKH. 


18. Use the Vigenére cipher with key BLUE to encrypt the 
message SNOWFALL. 


19. The ciphertext OIK YW VHBX was produced by encrypt- 
ing a plaintext message using the Vigenére cipher with 
key HOT. What is the plaintext message? 


20. Express the Vigenére cipher as a cryptosystem. 


To break a Vigenére cipher by recovering a plaintext message from 
the ciphertext message without having the key, the first step is to 
figure out the length of the key string. The second step is to figure 
out each character of the key string by determining the correspond- 
ing shift. Exercises 21 and 22 deal with these two aspects. 


21. Suppose that when a long string of text is encrypted using 
a Vigenére cipher, the same string is found in the ciphertext 
starting at several different positions. Explain how this infor- 
mation can be used to help determine the length of the key. 


22. Once the length of the key string of a Vigénere cipher is known, 
explain how to determine each of its characters. Assume that 
the plaintext is long enough so that the frequency of its let- 
ters is reasonably close to the frequency of letters in typical 
English text. 


* 23. Show that we can easily factor n when we know that n is the 
product of two primes, p and g, and we know the value of 
(p—Dq@-D. 

In Exercises 24—27 first express your answers without computing 
modular exponentiations. Then use a computational aid to com- 


plete these computations. 


24. Encrypt the message ATTACK using the RSA system with 
n = 43-59 and e = 13, translating each letter into integers 
and grouping together pairs of integers, as done in Example 


25. Encrypt the message UPLOAD using the RSA system with 
n = 53-61 and e = 17, translating each letter into integers 
and grouping together pairs of integers, as done in Example 
8. 


26. What is the original message encrypted using the RSA sys- 
tem with n = 53 - 61 and e = 17 if the encrypted message is 
3185 2038 2460 2550? (To decrypt, first find the decryption 
exponent d, which is the inverse of e = 17 modulo 52 - 60.) 


27. What is the original message encrypted using the RSA system 
with n = 43 - 59 and e = 13 if the encrypted message is 0667 
1947 0671? (To decrypt, first find the decryption exponent d 
which is the inverse of e = 13 modulo 42 - 58.) 
*28. Suppose that (n,e) is an RSA _ encryption key, 
with n= pq where p and q are large primes and 
gcd(e, (p — 1)(¢ — 1)) = 1. Furthermore, suppose that d 
is an inverse of e modulo (p — 1)(q — 1). Suppose that 
C = M° (mod pq). In the text we showed that RSA de- 
cryption, that is, the congruence C4 = M (mod pq) holds 
when gcd(M, pq) = 1. Show that this decryption congruence 
also holds when gcd(M, pq) > 1. [Hint: Use congruences 
modulo p and modulo g and apply the Chinese remainder 
theorem. |] 
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29. Describe the steps that Alice and Bob follow when they use 
the Diffie-Hellman key exchange protocol to generate a shared 
key. Assume that they use the prime p = 23 and take a = 5, 
which is a primitive root of 23, and that Alice selects kj = 8 
and Bob selects kz = 5. (You may want to use some compu- 
tational aid.) 


30. Describe the steps that Alice and Bob follow when they use 
the Diffie-Hellman key exchange protocol to generate a shared 
key. Assume that they use the prime p = 101 and take a = 2, 
which is a primitive root of 101, and that Alice selects kj = 7 
and Bob selects kz = 9. (You may want to use some compu- 
tational aid.) 


In Exercises 31-32 suppose that Alice and Bob have these 
public keys and corresponding private keys: (najice, Alice) = 
(2867, 7) = (61-47,7), datice = 1183 and (nBob, eBob) = 
(3127, 21) = (59-53, 21), dgob = 1149. First express your an- 
swers without carrying out the calculations. Then, using a com- 
putational aid, if available, perform the calculation to get the 
numerical answers. 


31. Alice wants to send to all her friends, including Bob, the mes- 
sage “SELL EVERYTHING” so that he knows that she sent 
it. What should she send to her friends, assuming she signs 
the message using the RSA cryptosystem. 


32. Alice wants to send to Bob the message “BUY NOW” so that 
he knows that she sent it and so that only Bob can read it. What 
should she send to Bob, assuming she signs the message and 
then encrypts it using Bob’s public key? 


33. We describe a basic key exchange protocol using private key 
cryptography upon which more sophisticated protocols for 
key exchange are based. Encryption within the protocol is 
done using a private key cryptosystem (such as AES) that is 
considered secure. The protocol involves three parties, Alice 
and Bob, who wish to exchange a key, and a trusted third party 
Cathy. Assume that Alice has a secret key kajice that only she 
and Cathy know, and Bob has a secret key kgop which only he 
and Cathy know. The protocol has three steps: 


(i) Alice sends the trusted third party Cathy the message “re- 
quest a shared key with Bob” encrypted using Alice’s key 
Kalice- 


(ii) Cathy sends back to Alice a key Kalice,Bob, Which she gen- 
erates, encrypted using the key Kajice, followed by this same 
key kalice,Bob, encrypted using Bob’s key, kgob. 


(iii) Alice sends to Bob the key kajice, Bob encrypted using kpob, 
known only to Bob and to Cathy. 


Explain why this protocol allows Alice and Bob to share 
the secret key kalice,Bob, Known only to them and to Cathy. 
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Key Terms and Results 


TERMS 


a | b (a divides b): there is an integer c such that b = ac 

a and b are congruent modulo m: m divides a — b 

modular arithmetic: arithmetic done modulo an integer 
m>2 

prime: an integer greater than 1 with exactly two positive 
integer divisors 

composite: an integer greater than 1 that is not prime 

Mersenne prime: a prime of the form 2? — 1, where p is prime 

gcd(a, b) (greatest common divisor of a and b): the largest 
integer that divides both a and b 

relatively prime integers: integers a and b such that 
gcd(a,b) = 1 

pairwise relatively prime integers: a set of integers with the 
property that every pair of these integers is relatively prime 

Icm(a, b) (least common multiple of a and b): the smallest 
positive integer that is divisible by both a and b 

a mod Bb: the remainder when the integer a is divided by the 
positive integer b 

a = b (mod m) (a is congruent to b modulo m): a — b is 
divisible by m 

n = (agag_1...@ a )p: the base b representation of n 

binary representation: the base 2 representation of an integer 

octal representation: the base 8 representation of an integer 
hexadecimal representation: the base 16 representation of 
an integer 

linear combination of a and b with integer coefficients: an 
expression of the form sa + tb, where s and t are integers 

Bézout coefficients of a and b: integers s and t such that the 
Bézout identity sa + tb = gcd(a, b) holds 

inverse of a modulo m: an integer a such that da = 1 (mod m) 

linear congruence: a congruence of the form ax = b(modm), 
where x is an integer variable 

pseudoprime to the base b: a composite integer n such that 
b"—! = 1 (mod n) 

Carmichael number: a composite integer n such that n is a 
pseudoprime to the base b for all positive integers b with 
gcd(b,n) = 1 

primitive root of a prime p: an integer r in Z,, such that every 
integer not divisible by p is congruent modulo p to a power 
of r 

discrete logarithm of a to the base r modulo p: the integer e 
with 0 < e < p—1 such that r° = a (mod p) 

encryption: the process of making a message secret 

decryption: the process of returning a secret message to its 
original form 

encryption key: a value that determines which of a family of 
encryption functions is to be used 

shift cipher: a cipher that encrypts the plaintext letter p as 
(p +k) mod m for an integer k 

affine cipher: a cipher that encrypts the plaintext letter p as 
(ap + b) mod m for integers a and b with gcd(a, 26) = 1 

character cipher: a cipher that encrypts characters one by one 

block cipher: a cipher that encrypts blocks of characters of a 
fixed size 


crytanalysis: the process of recovering the plaintext from ci- 
phertext without knowledge of the encryption method, or 
with knowledge of the encryption method, but not the key 

cryptosystem: a five-tuple (P,C, K, E€, D) where P is the set 
of plaintext messages, C is the set of ciphertext messages, 
K is the set of keys, € is the set of encryption functions, 
and D is the set of decryption functions 

private key encryption: encryption where both encryption 
keys and decryption keys must be kept secret 

public key encryption: encryption where encryption keys are 
public knowledge, but decryption keys are kept secret 

RSA cryptosystem: the cryptosystem where P and C are 
both Z26, K is the set of pairs k = (n, e) where n = pq 
where p and q are large primes and e is a positive integer, 
Ex(p) = p* mod n, and Dx(c) = c4 mod n where d is the 
inverse of e modulo (p — 1)(g — 1) 

key exchange protocol: a protocol used for two parties to 
generate a shared key 

digital signature: a method that a recipient can use to deter- 
mine that the purported sender of a message actually sent 
the message 


RESULTS 


division algorithm: Let a and d be integers with d positive. 
Then there are unique integers g andr withO < r <d such 
thata =dq+r. 

Let b be an integer greater than 1. Then if n is a pos- 
itive integer, it can be expressed uniquely in the form 
n = agb* + ag_1b'! +--+ + arb + ao. 

The algorithm for finding the base b expansion of an integer 
(see Algorithm 1 in Section 4.2) 

The conventional algorithms for addition and multiplication 
of integers (given in Section 4.2) 

The modular exponentiation algorithm (see Algorithm 5 in 
Section 4.2) 

Euclidean algorithm: for finding greatest common divisors 
by successively using the division algorithm (see Algorithm 
1 in Section 4.3) 

Bézout’s theorem: If a and b are positive integers, then 
gcd(a, b) is a linear combination of a and b. 

sieve of Eratosthenes: A procedure for finding all primes not 
exceeding a specified number n, described in Section 4.3 

fundamental theorem of arithmetic: Every positive integer 
can be written uniquely as the product of primes, where the 
prime factors are written in order of increasing size. 

Ifa and Dare positive integers, thenab = gcd(a, b)-lcm(a, b). 

If m is a positive integer and gcd(a,m) = 1, then a has a 
unique inverse modulo m. 

Chinese remainder theorem: A system of linear congruences 
modulo pairwise relatively prime integers has a unique so- 
lution modulo the product of these moduli. 

Fermat’s little theorem: If p is prime and p Ja, then 
a?—! = 1 (mod p). 


Review Questions 


1. 


Find 210 div 17 and 210 mod 17. 


2. a) Define what it means for a and b to be congruent mod- 


“I 


10. 


ulo 7. 

b) Which pairs of the integers —11, —8, —7, —1, 0, 3, 
and 17 are congruent modulo 7? 

c) Show that if a and b are congruent modulo 7, then 
10a + 13 and —4b + 20 are also congruent modulo 7. 


. Show that if a = b (mod m) and c =d (mod m), then 


a+c=b+d(modm). 


. Describe a procedure for converting decimal (base 10) 


expansions of integers into hexadecimal expansions. 


. Convert (1101 1001 0101 1011). to octal and hexadeci- 


mal representations. 


. Convert (7206)g and (AOEB)1¢ to a binary representation. 
. State the fundamental theorem of arithmetic. 
. a) Describe a procedure for finding the prime factoriza- 


tion of an integer. 
b) Use this procedure to find the prime factorization of 
80,707. 


. a) Define the greatest common divisor of two integers. 


b) Describe at least three different ways to find the great- 
est common divisor of two integers. When does each 
method work best? 

c) Find the greatest common divisor of 1,234,567 and 
7,654,321. 

d) Find the greatest common divisor of 2735577911 and 
aS 1g, 

a) How can you find a linear combination (with integer 
coefficients) of two integers that equals their greatest 
common divisor? 


Supplementary Exercises 


1. 


The odometer on a car goes to up 100,000 miles. The 
present owner of a car bought it when the odometer read 
43,179 miles. He now wants to sell it; when you examine 
the car for possible purchase, you notice that the odome- 
ter reads 89,697 miles. What can you conclude about how 
many miles he drove the car, assuming that the odometer 
always worked correctly? 


. a) Explain why n div 7 equals the number of complete 


weeks in n days. 
b) Explain why n div 24 equals the number of complete 
days in n hours. 


. Find four numbers congruent to 5 modulo 17. 


4. Show that if a and d are positive integers, then there are 


#5, 


integers g andr such thata = dq +rwhere—d/2 <r < 
d/2. 


Show that if ac=bc(modm), where a,b,c, and 
m are integers with m > 2, and d = gcd(m,c), then 
a=b(modm/d). 


. Show that the sum of the squares of two odd integers 


cannot be the square of an integer. 


11. 


12. 


13. 


14. 
15. 
16. 
17. 


18. 


19. 


20. 


10. 


11. 


12. 


13. 
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b) Express gcd(84, 119) as a linear combination of 84 
and 119. 

a) What does it mean for @ to be an inverse of 
a modulo m? 

b) How can you find an inverse of a modulo m when m 
is a positive integer and gcd(a, m) = 1? 

c) Find an inverse of 7 modulo 19. 

a) How can an inverse of a modulo m be used to solve the 
congruence ax = b (mod m) when gcd(a, m) = 1? 

b) Solve the linear congruence 7x = 13 (mod 19). 

a) State the Chinese remainder theorem. 

b) Find the solutions to the system x =1 (mod 4), 
x =2 (mod 5), and x = 3 (mod 7). 

Suppose that 2”—! = 1 (mod n). Is n necessarily prime? 

Use Fermat’s little theorem to evaluate 97° mod 19. 

Explain how the check digit is found for a 10-digit ISBN. 


Encrypt the message APPLES AND ORANGES using a 

shift cipher with key k = 13. 

a) What is the difference between a public key and a pri- 
vate key cryptosystem? 

b) Explain why using shift ciphers is a private key sys- 
tem. 

c) Explain why the RSA cryptosystem is a public key 
system. 

Explain how encryption and decryption are done in the 

RSA cryptosystem. 

Describe how two parties can share a secret key using the 

Diffie-Hellman key exchange protocol. 


. Show that if n? +1 is a perfect square, where n is an 


integer, then n is even. 


. Prove that there are no solutions in integers x and y to 


the equation x7 — 5y” = 2. [Hint: Consider this equation 
modulo 5.] 


. Develop a test for divisibility of a positive integer n by 8 


based on the binary expansion of n. 


Develop a test for divisibility of a positive integer n by 3 
based on the binary expansion of n. 


Devise an algorithm for guessing a number between 1 
and 2” — 1 by successively guessing each bit in its binary 
expansion. 


Determine the complexity, in terms of the number of 
guesses, needed to determine a number between 1 and 
2” — | by successively guessing the bits in its binary ex- 
pansion. 


Show that an integer is divisible by 9 if and only if the 
sum of its decimal digits is divisible by 9. 
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** 14, 


15. 


16. 


17. 


18. 


*19, 


20. 
21. 


22. 


* 23. 


*24, 


25. 


26. 


27. 


28. 


29. 


Show that if a and b are positive irrational numbers such 
that 1/a + 1/b = 1, then every positive integer can be 
uniquely expressed as either [ka| or |kb| for some posi- 
tive integer k. 


Prove there are infinitely many primes by showing that 
Q, =n!+1 must have a prime factor greater than n 
whenever n is a positive integer. 


Find a positive integer n for which Q, =n! + 1 is not 
prime. 


Use Dirichlet’s theorem, which states there are infinitely 
many primes in every arithmetic progression ak +b 
where gcd(a, b) = 1, to show that there are infinitely 
many primes that have a decimal expansion ending 
with a 1. 


Prove that if n is a positive integer such that the sum of 
the divisors of n isn + 1, then n is prime. 


Show that every integer greater than 11 is the sum of two 
composite integers. 


Find the five smallest consecutive composite integers. 


Show that Goldbach’s conjecture, which states that ev- 
ery even integer greater than 2 is the sum of two primes, 
is equivalent to the statement that every integer greater 
than 5 is the sum of three primes. 


Find an arithmetic progression of length six beginning 
with 7 that contains only primes. 


Prove that if f(x) is a nonconstant polynomial with inte- 
ger coefficients, then there is an integer y such that f(y) is 
composite. [Hint: Assume that f (xo) = p is prime. Show 
that p divides f (xo + kp) for all integers k. Obtain a con- 
tradiction of the fact that a polynomial of degree n, where 
n > 1, takes on each value at most n times.] 


How many zeros are at the end of the binary expansion 
of 1009)? 


Use the Euclidean algorithm to find the greatest common 
divisor of 10,223 and 33,341. 


How many divisions are required to find gcd(144, 233) 
using the Euclidean algorithm? 


Find gcd(2n + 1, 3n + 2), where n is a positive integer. 
[Hint: Use the Euclidean algorithm. ] 


a) Show that if a and b are positive inte- 
gers with a>b, then gcd(a,b)=a if a=b, 
gced(a,b) = 2 gcd(a/2, b/2) if a and b are even, 
gcd(a, b) = gcd(a/2, b) if a is even and b is odd, and 
gcd(a, b) = gcd(a — b, b) if both a and b are odd. 

b) Explain how to use (a) to construct an algorithm for 
computing the greatest common divisor of two posi- 
tive integers that uses only comparisons, subtractions, 
and shifts of binary expansions, without using any 
divisions. 

c) Find ged(1202, 4848) using this algorithm. 


Adapt the proof that there are infinitely many primes (The- 
orem 3 in Section 4.3) to show that are infinitely many 
primes in the arithmetic progression 6k + 5,k = 1,2,.... 


30. 


31. 


32. 


Explain why you cannot directly adapt the proof that there 
are infinitely many primes (Theorem 3 in Section 4.3) to 
show that there are infinitely many primes in the arith- 
metic progression 3k + 1,k = 1,2,.... 

Explain why you cannot directly adapt the proof that there 
are infinitely many primes (Theorem 3 in Section 4.3) to 
show that are infinitely many primes in the arithmetic 
progression 4k + 1,k = 1,2,.... 

Show that if the smallest prime factor p of the positive 


integer n is larger than ~/n, then n/p is prime or equal to 
1. 


A set of integers is called mutually relatively prime if the 
greatest common divisor of these integers is 1. 


33. 


34. 


* 35. 
36. 


37. 


* 38. 


39. 


40. 


41. 


Determine whether the integers in each of these sets are 
mutually relatively prime. 

a) 8, 10, 12 b) 12,15, 25 

ce) 15,21, 28 d) 21, 24, 28, 32 

Find a set of four mutually relatively prime integers such 
that no two of them are relatively prime. 


For which positive integers n is n* + 4” prime? 

Show that the system of congruences x = 2 (mod 6) and 

x = 3 (mod 9) has no solutions. 

Find all solutions of the system of congruences x = 

4 (mod 6) and x = 13 (mod 15). 

a) Show that the system of congruences x =a;(modm 1) 
and x = a2 (mod m2), where aj, a2, m1, and mz are 
integers with m; > 0 and m2 > 0, has a solution if 
and only if gcd(m1, m2) | a1 — a2. 

b) Show that if the system in part (a) has a solution, then 
it is unique modulo Icm(m 1, m2). 

Prove that 30 divides n° —n for every nonnegative 

integer n. 


Prove that n!? — 1 is divisible by 35 for every integer n 
for which gcd(n, 35) = 1. 

Show that if p and gq are distinct prime numbers, then 
pi! +q?—! = 1 (mod pq). 


The check digit a3 for an ISBN-13 with initial digits 
a\a2...a,2 iS determined by the congruence (a; + a3 + 
+++ + ay3) + 3(a@2 +a44+--- +42) = 0 (mod 10). 


42. 


43. 


44. 


Determine whether each of these 13-digit numbers is a 
valid ISBN-13. 

a) 978-0-073-20679-1 

b) 978-0-45424-521-1 

c) 978-3-16-148410-0 

d) 978-0-201-10179-9 

Show that the check digit of an ISBN-13 can always detect 
a single error. 


Show that there are transpositions of two digits that are 
not detected by an ISBN-13. 


A routing transit number (RTN) is a bank code used in 
the United States which appears on the bottom of checks. 
The most common form of an RTN has nine digits, where 
the last digit is a check digit. If djdz...dg is a valid RTN, 


the congruence 3(d; + d4 + d7) + 7(do + ds + dg) + (d3 + 

do + dy) = 0 (mod 10) must hold. 

45. Show that if djd2...do is a valid RTN, then dy = 7(d) + 
d4 + d7) + 3(d2 + ds + dg) + 9(d3 + do) mod 10. Fur- 
thermore, use this formula to find the check digit that 
follows the eight digits 11100002 in a valid RTN. 

46. Show that the check digit of an RTN can detect all single 
errors and determine which transposition errors an RTN 
check digit can catch and which ones it cannot catch. 

47. The encrypted version of a message is LIMKG MG- 
MXF QEXMW. If it was encrypted using the affine ci- 
pher f(p) = (7p + 10) mod 26, what was the original 
message? 

Autokey ciphers are ciphers where the nth letter of the plain- 

text is shifted by the numerical equivalent of the nth letter of 

a keystream. The keystream begins with a seed letter; its sub- 

sequent letters are constructed using either the plaintext or the 

ciphertext. When the plaintext is used, each character of the 


Computer Projects 


Write programs with these inputs and outputs. 


1. Given integers n and b, each greater than 1, find the base 
b expansion of this integer. 

2. Given the positive integers a, b, and m with m > 1, find 
a’ mod m. 

3. Given a positive integer, find the Cantor expansion of this 
integer (see the preamble to Exercise 48 of Section 4.2). 

4. Given a positive integer, determine whether it is prime 
using trial division. 

5. Given a positive integer, find the prime factorization of 
this integer. 

6. Given two positive integers, find their greatest common 
divisor using the Euclidean algorithm. 

7. Given two positive integers, find their least common mul- 
tiple. 

8. Given positive integers a and b, find Bézout coefficients 
s and t of a and b. 

9. Given relatively prime positive integers a and b, find an 
inverse of a modulo b. 

10. Given n linear congruences modulo pairwise relatively 
prime moduli, find the simultaneous solution of these con- 
gruences modulo the product of these moduli. 

11. Givena positive integer N, a modulus m, a multiplier a, an 
increment c, andaseed xp, whereO < a <m,0<c<™m, 
and 0 < x9 <_m, generate the sequence of N pseudo- 
random numbers using the linear congruential generator 
Xn41 = (ax, +c) mod m. 

12. Given a set of identification numbers, use a hash func- 
tion to assign them to memory locations where there are 
k memory locations. 

13. Compute the check digit when given the first nine digits 
of an ISBN-10. 
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keystream, after the first, is the previous letter of the plaintext. 
When the ciphertext is used, each subsequent character of the 
keystream, after the first, is the previous letter of the ciphertext 
computed so far. In both cases, plaintext letters are encrypted 
by shifting each character by the numerical equivalent of the 
corresponding keystream letter. 


48. Use the autokey cipher to encrypt the message NOW IS 
THE TIME TO DECIDE (ignoring spaces) using 
a) the keystream with seed X followed by letters of the 
plaintext. 
b) the keystream with seed X followed by letters of the 
ciphertext. 


49. Use the autokey cipher to encrypt the message THE 
DREAM OF REASON (ignoring spaces) using 
a) the keystream with seed X followed by letters of the 
plaintext. 
b) the keystream with seed X followed by letters of the 
ciphertext. 


14. Given a message and a positive integer k less than 26, 
encrypt this message using the shift cipher with key k; 
and given a message encrypted using a shift cipher with 
key k, decrypt this message. 


15. Given a message and positive integers a and b less than 
26 with gcd(a, 26), encrypt this message using an affine 
cipher with key (a, b); and given a message encrypted us- 
ing the affine cipher with key (a, b), decrypt this message, 
by first finding the decryption key and then applying the 
appropriate decryption transformation. 


16. Find the original plaintext message from the ciphertext 
message produced by encrypting the plaintext message 
using a shift cipher. Do this using a frequency count of 
letters in the ciphertext. 


*17. Construct a valid RSA encryption key by finding two 
primes p and q with 200 digits each and an integer e > 1 
relatively prime to (p — 1)(g — 1). 


18. Given a message and an integer n = pq where p and 
q are odd primes and an integer e > 1 relatively prime 
to (p — 1)(q — 1), encrypt the message using the RSA 
cryptosystem with key (n, e). 


19. Given a valid RSA key (n, e), and the primes p and q 
with n = pq, find the associated decryption key d. 


20. Given a message encrypted using the RSA cryptosystem 
with key (n, e) and the associated decryption key d, de- 
crypt this message. 

21. Generate a shared key using the Diffie-Hellman key ex- 
change protocol. 

22. Given the RSA public and private keys of two parties, 


send a signed secret message from one of the parties to 
the other. 
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Computations and Explorations 


Use a computational program or programs you have written to do these exercises. 


. Determine whether 2? — 1 is prime for each of the primes 
not exceeding 100. 

. Test a range of large Mersenne numbers 2? — | to de- 
termine whether they are prime. (You may want to use 
software from the GIMPS project.) 

. Determine whether Q, = pip2--:-Pn+1 is prime 
where pi, P2,.--, Pn are the n smallest primes, for as 
many positive integer n as possible. 

. Look for polynomials in one variables whose values at 
long runs of consecutive integers are all primes. 

. Find as many primes of the form n? + 1 where n is a pos- 
itive integer as you can. It is not known whether there are 
infinitely many such primes. 


Writing Projects 


Respond to these with essays using outside sources. 


. Describe the Lucas—Lehmer test for determining whether 
a Mersenne number is prime. Discuss the progress of the 
GIMPS project in finding Mersenne primes using this test. 


. Explain how probabilistic primality tests are used in prac- 
tice to produce extremely large numbers that are almost 
certainly prime. Do such tests have any potential draw- 
backs? 


. The question of whether there are infinitely many 
Carmichael numbers was solved recently after being open 
for more than 75 years. Describe the ingredients that went 
into the proof that there are infinitely many such numbers. 


. Summarize the current status of factoring algorithms in 
terms of their complexity and the size of numbers that can 
currently be factored. When do you think that it will be 
feasible to factor 200-digit numbers? 


. Describe the algorithms that are actually used by modern 
computers to add, subtract, multiply, and divide positive 
integers. 


. Describe the history of the Chinese remainder theorem. 
Describe some of the relevant problems posed in Chi- 
nese and Hindu writings and how the Chinese remainder 
theorem applies to them. 


. When are the numbers of a sequence truly random num- 
bers, and not pseudorandom? What shortcomings have 
been observed in simulations and experiments in which 
pseudorandom numbers have been used? What are the 
properties that pseudorandom numbers can have that ran- 
dom numbers should not have? 


10. 


11. 


12. 


13. 


*14, 


15. 


. Find 10 different primes each with 100 digits. 
. How many primes are there less than 1,000,000, less than 


10,000,000, and less than 100,000,000? Can you propose 
an estimate for the number of primes less than x where x 
is a positive integer? 


. Find a prime factor of each of 10 different 20-digit odd 


integers, selected at random. Keep track of how long it 
takes to find a factor of each of these integers. Do the 
same thing for 10 different 30-digit odd integers, 10 dif- 
ferent 40-digit odd integers, and so on, continuing as long 
as possible. 


. Find all pseudoprimes to the base 2 that do not exceed 


10,000. 


. Explain how a check digit is found for an International 


Bank Account Number (IBAN) and discuss the types of 
errors that can be found using this check digit. 


. Describe the Luhn algorithm for finding the check digit 


of a credit card number and discuss the types of errors 
that can found using this check digit. 


Show how a congruence can be used to tell the day of the 
week for any given date. 

Describe how public key cryptography is being applied. 
Are the ways it is applied secure given the status of fac- 
toring algorithms? Will information kept secure using 
public key cryptography become insecure in the future? 


Describe how public key cryptography can be used to 
produce signed secret messages so that the recipient is 
relatively sure the message was sent by the person ex- 
pected to have sent it. 


Describe the Rabin public key cryptosystem, explaining 
how to encrypt and how to decrypt messages and why it 
is suitable for use as a public key cryptosystem. 


Explain why it would not be suitable to use p, where 
Pp is a large prime, as the modulus for encryption in the 
RSA cryptosystem. That is, explain how someone could, 
without excessive computation, find a private key from 
the corresponding public key if the modulus were a large 
prime, rather than the product of two large primes. 

Explain what is meant by a cryptographic hash function? 


What are the important properties such a function must 
have? 


5.1 Mathematical 
Induction 


5.2 Strong 
Induction and 
Well-Ordering 


5.3 Recursive 
Definitions and 
Structural 
Induction 


5.4 Recursive 
Algorithms 


5.5 Program 
Correctness 


Induction and Recursion 


Mz mathematical statements assert that a property is true for all positive integers. 
Examples of such statements are that for every positive integer n: n! <n", n> —n is 
divisible by 3; a set with n elements has 2” subsets; and the sum of the first n positive integers 
is n(n + 1)/2. A major goal of this chapter, and the book, is to give the student a thorough 
understanding of mathematical induction, which is used to prove results of this kind. 

Proofs using mathematical induction have two parts. First, they show that the statement 
holds for the positive integer 1. Second, they show that if the statement holds for a positive 
integer then it must also hold for the next larger integer. Mathematical induction is based on the 
rule of inference that tells us that if P(1) and Vk(P(k) — P(k + 1)) are true for the domain of 
positive integers, then Vn P (n) is true. Mathematical induction can be used to prove a tremendous 
variety of results. Understanding how to read and construct proofs by mathematical induction 
is a key goal of learning discrete mathematics. 

In Chapter 2 we explicitly defined sets and functions. That is, we described sets by listing 
their elements or by giving some property that characterizes these elements. We gave formulae 
for the values of functions. There is another important way to define such objects, based on 
mathematical induction. To define functions, some initial terms are specified, and a rule is 
given for finding subsequent values from values already known. (We briefly touched on this 
sort of definition in Chapter 2 when we showed how sequences can be defined using recurrence 
relations.) Sets can be defined by listing some of their elements and giving rules for constructing 
elements from those already known to be in the set. Such definitions, called recursive definitions, 
are used throughout discrete mathematics and computer science. Once we have defined a set 
recursively, we can use a proof method called structural induction to prove results about this set. 

When a procedure is specified for solving a problem, this procedure must always solve 
the problem correctly. Just testing to see that the correct result is obtained for a set of input 
values does not show that the procedure always works correctly. The correctness of a procedure 
can be guaranteed only by proving that it always yields the correct result. The final section of 
this chapter contains an introduction to the techniques of program verification. This is a formal 
technique to verify that procedures are correct. Program verification serves as the basis for 
attempts under way to prove in a mechanical fashion that programs are correct. 


Mathematical Induction 


Introduction 


Suppose that we have an infinite ladder, as shown in Figure |, and we want to know whether 
we can reach every step on this ladder. We know two things: 


1. We can reach the first rung of the ladder. 
2. If we can reach a particular rung of the ladder, then we can reach the next rung. 


Can we conclude that we can reach every rung? By (1), we know that we can reach the first 
rung of the ladder. Moreover, because we can reach the first rung, by (2), we can also reach the 
second rung; it is the next rung after the first rung. Applying (2) again, because we can reach 
the second rung, we can also reach the third rung. Continuing in this way, we can show that we 
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Assessment 


We can reach step k + 1 if 
we can reach step k 


We can reach 
step | 


FIGURE 1 Climbing an Infinite Ladder. 


can reach the fourth rung, the fifth rung, and so on. For example, after 100 uses of (2), we know 
that we can reach the 101st rung. But can we conclude that we are able to reach every rung 
of this infinite ladder? The answer is yes, something we can verify using an important proof 
technique called mathematical induction. That is, we can show that P(n) is true for every 
positive integer n, where P(n) is the statement that we can reach the nth rung of the ladder. 

Mathematical induction is an extremely important proof technique that can be used to prove 
assertions of this type. As we will see in this section and in subsequent sections of this chapter 
and later chapters, mathematical induction is used extensively to prove results about a large 
variety of discrete objects. For example, it is used to prove results about the complexity of 
algorithms, the correctness of certain types of computer programs, theorems about graphs and 
trees, as well as a wide range of identities and inequalities. 

In this section, we will describe how mathematical induction can be used and why it is a 
valid proof technique. It is extremely important to note that mathematical induction can be used 
only to prove results obtained in some other way. It is not a tool for discovering formulae or 
theorems. 


Mathematical Induction 


In general, mathematical induction * can be used to prove statements that assert that P(n) is 
true for all positive integers n, where P (7) is a propositional function. A proof by mathematical 


*Unfortunately, using the terminology “mathematical induction” clashes with the terminology used to describe different types 
of reasoning. In logic, deductive reasoning uses rules of inference to draw conclusions from premises, whereas inductive 
reasoning makes conclusions only supported, but not ensured, by evidence. Mathematical proofs, including arguments that 
use mathematical induction, are deductive, not inductive. 


Links 
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induction has two parts, a basis step, where we show that P(1) is true, and an inductive step, 
where we show that for all positive integers k, if P(k) is true, then P(k + 1) is true. 


PRINCIPLE OF MATHEMATICAL INDUCTION To prove that P(n) is true for all 
positive integers n, where P(n) is a propositional function, we complete two steps: 


BASIS STEP: We verify that P (1) is true. 


INDUCTIVE STEP: We show that the conditional statement P(k) — P(k + 1) is true for 
all positive integers k. 


To complete the inductive step of a proof using the principle of mathematical induction, we 
assume that P(x) is true for an arbitrary positive integer k and show that under this assumption, 
P(k + 1) must also be true. The assumption that P (x) is true is called the inductive hypothesis. 
Once we complete both steps in a proof by mathematical induction, we have shown that P (1) is 
true for all positive integers, that is, we have shown that Vn P (n) is true where the quantification 
is over the set of positive integers. In the inductive step, we show that Vk(P(k) — P(k+ 1)) 
is true, where again, the domain is the set of positive integers. 
Expressed as a rule of inference, this proof technique can be stated as 


(P(A) AVK(P(k) > P(K+1))) > VnP(n), 


when the domain is the set of positive integers. Because mathematical induction is such an 
important technique, it is worthwhile to explain in detail the steps of a proof using this technique. 
The first thing we do to prove that P (7) is true for all positive integers n is to show that P(1) is 
true. This amounts to showing that the particular statement obtained when n is replaced by | in 
P(n) is true. Then we must show that P(k) > P(k + 1) is true for every positive integer k. To 
prove that this conditional statement is true for every positive integer k, we need to show that 
P(k + 1) cannot be false when P(k) is true. This can be accomplished by assuming that P(k) 
is true and showing that under this hypothesis P(k + 1) must also be true. 


Remark: Ina proof by mathematical induction it is not assumed that P (k) is true for all positive 
integers! It is only shown that if it is assumed that P(k) is true, then P(k + 1) is also true. Thus, 
a proof by mathematical induction is not a case of begging the question, or circular reasoning. 


When we use mathematical induction to prove a theorem, we first show that P(1) is true. Then 
we know that P(2) is true, because P(1) implies P(2). Further, we know that P(3) is true, 
because P(2) implies P(3). Continuing along these lines, we see that P(n) is true for every 
positive integer n. 


HISTORICAL NOTE The first known use of mathematical induction is in the work of the sixteenth-century 
mathematician Francesco Maurolico (1494-1575). Maurolico wrote extensively on the works of classical 
mathematics and made many contributions to geometry and optics. In his book Arithmeticorum Libri Duo, 
Maurolico presented a variety of properties of the integers together with proofs of these properties. To prove 
some of these properties, he devised the method of mathematical induction. His first use of mathematical 
induction in this book was to prove that the sum of the first n odd positive integers equals n. Augustus De 
Morgan is credited with the first presentation in 1838 of formal proofs using mathematical induction, as well 
as introducing the terminology “mathematical induction.” Maurolico’s proofs were informal and he never used 
the word “induction.” See [Gu11] to learn more about the history of the method of mathematical induction. 
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FIGURE 2 Illustrating How Mathematical Induction Works Using Dominoes. 


WAYS TO REMEMBER HOW MATHEMATICAL INDUCTION WORKS Thinking of 
the infinite ladder and the rules for reaching steps can help you remember how mathematical 
induction works. Note that statements (1) and (2) for the infinite ladder are exactly the basis 
step and inductive step, respectively, of the proof that P(v) is true for all positive integers n, 
where P(n) is the statement that we can reach the nth rung of the ladder. Consequently, we can 
invoke mathematical induction to conclude that we can reach every rung. 

Another way to illustrate the principle of mathematical induction is to consider an infinite 
row of dominoes, labeled 1, 2,3,...,”,..., where each domino is standing up. Let P(n) be 
the proposition that domino n is knocked over. If the first domino is knocked over—i.e., if P (1) 
is true—and if, whenever the kth domino is knocked over, it also knocks the (k + 1)st domino 
over—i.e., if P(k) > P(k + 1) is true for all positive integers k—then all the dominoes are 
knocked over. This is illustrated in Figure 2. 


Why Mathematical Induction is Valid 


Why is mathematical induction a valid proof technique? The reason comes from the well- 
ordering property, listed in Appendix 1, as an axiom for the set of positive integers, which 
states that every nonempty subset of the set of positive integers has a least element. So, suppose 
we know that P(1) is true and that the proposition P(k) — P(k + 1) is true for all positive 
integers k. To show that P(n) must be true for all positive integers n, assume that there is at 
least one positive integer for which P(n) is false. Then the set S of positive integers for which 
P(n) is false is nonempty. Thus, by the well-ordering property, S has a least element, which 
will be denoted by m. We know that m cannot be 1, because P(1) is true. Because m is positive 
and greater than 1, m — | is a positive integer. Furthermore, because m — | is less than m, it is 
not in S, so P(m — 1) must be true. Because the conditional statement P(m — 1) — P(m) is 
also true, it must be the case that P(m) is true. This contradicts the choice of m. Hence, P(n) 
must be true for every positive integer n. 


The Good and the Bad of Mathematical Induction 


An important point needs to be made about mathematical induction before we commence a 
study of its use. The good thing about mathematical induction is that it can be used to prove 


You can prove 
a theorem by 
mathematical 
induction 
even if you do 
not have the 
slightest idea 
why it is true! 


Links 


Look for the w symbol to 
see where the inductive 
hypothesis is used. 
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a conjecture once it is has been made (and is true). The bad thing about it is that it cannot 
be used to find new theorems. Mathematicians sometimes find proofs by mathematical in- 
duction unsatisfying because they do not provide insights as to why theorems are true. Many 
theorems can be proved in many ways, including by mathematical induction. Proofs of these the- 
orems by methods other than mathematical induction are often preferred because of the insights 
they bring. 


Examples of Proofs by Mathematical Induction 


Many theorems assert that P(n) is true for all positive integers n, where P(n) 1s a propositional 
function. Mathematical induction is a technique for proving theorems of this kind. In other words, 
mathematical induction can be used to prove statements of the form Vn P(n), where the domain 
is the set of positive integers. Mathematical induction can be used to prove an extremely wide 
variety of theorems, each of which is a statement of this form. (Remember, many mathematical 
assertions include an implicit universal quantifier. The statement “if n is a positive integer, then 
n> — nis divisible by 3” is an example of this. Making the implicit universal quantifier explicit 
yields the statement “for every positive integer n, n> — n is divisible by 3.) 

We will use how theorems are proved using mathematical induction. The theorems we will 
prove include summation formulae, inequalities, identities for combinations of sets, divisibility 
results, theorems about algorithms, and some other creative results. In this section and in later 
sections, we will employ mathematical induction to prove many other types of results, including 
the correctness of computer programs and algorithms. Mathematical induction can be used to 
prove a wide variety of theorems, not just summation formulae, inequalities, and other types of 
examples we illustrate here. (For proofs by mathematical induction of many more interesting 
and diverse results, see the Handbook of Mathematical Induction by David Gunderson [Gul 1]. 
This book is part of the extensive CRC Series in Discrete Mathematics, many of which may be 
of interest to readers. The author is the Series Editor of these books). 

Note that there are many opportunities for errors in induction proofs. We will describe some 
incorrect proofs by mathematical induction at the end of this section and in the exercises. To 
avoid making errors in proofs by mathematical induction, try to follow the guidelines for such 
proofs given at the end of this section. 


SEEING WHERETHE INDUCTIVEHYPOTHESISISUSED_  Tohelp the reader understand 
each of the mathematical induction proofs in this section, we will note where the inductive 
hypothesis is used. We indicate this use in three different ways: by explicit mention in the text, 
by inserting the acronym IH (for inductive hypothesis) over an equals sign or a sign for an 
inequality, or by specifying the inductive hypothesis as the reason for a step in a multi-line 
display. 


PROVING SUMMATION FORMULAE _ We begin by using mathematical induction to prove 
several summation formulae. As we will see, mathematical induction is particularly well suited 
for proving that such formulae are valid. However, summation formulae can be proven in other 
ways. This is not surprising because there are often different ways to prove a theorem. The major 
disadvantage of using mathematical induction to prove a summation formula is that you cannot 
use it to derive this formula. That is, you must already have the formula before you attempt to 
prove it by mathematical induction. 

Examples 1-4 illustrate how to use mathematical induction to prove summation formulae. 
The first summation formula we will prove by mathematical induction, in Example 1, is a closed 
formula for the sum of the smallest n positive integers. 
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EXAMPLE 1 
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EXAMPLE 2 


Show that if 7 is a positive integer, then 


n(n + 1) 


1424... = 
a i 5 


Solution: Let P(n) be the proposition that the sum of the first n positive integers, 1 + 2+---n= 
a is n(n + 1)/2. We must do two things to prove that P(n) is true forn = 1,2,3,.... 
Namely, we must show that P (1) is true and that the conditional statement P(k) implies P(k + 1) 
is true fork = 1,2,3,.... 


1041 
BASIS STEP: P(1) is true, because 1 = ie?) (The left-hand side of this equation is 1 


because | is the sum of the first positive integer. The right-hand side is found by substituting | 
for n inn(n + 1)/2.) 


INDUCTIVE STEP: For the inductive hypothesis we assume that P(k) holds for an arbitrary 
positive integer k. That is, we assume that 


k(k + 1) 


1424+---+k= 
2 
Under this assumption, it must be shown that P(k + 1) is true, namely, that 


KK+IIKA+D +1) &+DK+2) 
2 ~ 2 


1424+--+k+(k+)= 


is also true. When we add k + | to both sides of the equation in P(k), we obtain 


k(k +1 

1t24- tk Q+y SETS 
— kkK+1 +2 +1) 
~ 2 
kt DK +2) 
=. 


+(k+1) 


This last equation shows that P(k + 1) is true under the assumption that P(x) is true. This 
completes the inductive step. 


We have completed the basis step and the inductive step, so by mathematical induction we 
know that P (7) is true for all positive integers n. That is, we have proven that] +2+---+n= 
n(n + 1)/2 for all positive integers n. < 


As we noted, mathematical induction is not a tool for finding theorems about all positive 
integers. Rather, it is a proof method for proving such results once they are conjectured. In 
Example 2, using mathematical induction to prove a summation formula, we will both formulate 
and then prove a conjecture. 


Conjecture a formula for the sum of the first 7 positive odd integers. Then prove your conjecture 
using mathematical induction. 


Solution: The sums of the first n positive odd integers for n = 1, 2,3, 4,5 are 


1=1, 14+3=4, 14+3+5=9, 
14+34+54+7=16, 14+34+54+7+9=25. 
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From these values it is reasonable to conjecture that the sum of the first n positive odd integers 
is n”, that is, 1 +3+5+---+(2n — 1) =n?. We need a method to prove that this conjecture 
is correct, if in fact it is. 

Let P(n) denote the proposition that the sum of the first n odd positive integers is n?. Our 
conjecture is that P(7) is true for all positive integers. To use mathematical induction to prove 
this conjecture, we must first complete the basis step; that is, we must show that P(1) is true. 
Then we must carry out the inductive step; that is, we must show that P(A + 1) is true when 
P(k) is assumed to be true. We now attempt to complete these two steps. 


BASIS STEP: P(\) states that the sum of the first one odd positive integer is 1°. This is true 
because the sum of the first odd positive integer is 1. The basis step is complete. 


INDUCTIVE STEP: To complete the inductive step we must show that the proposition 
P(k) ~ P(k + 1) is true for every positive integer k. To do this, we first assume the inductive 
hypothesis. The inductive hypothesis is the statement that P(x) is true for an arbitrary positive 
integer k, that is, 


14+3454---+(Qk-D=Pk. 


(Note that the kth odd positive integer is (2k — 1), because this integer is obtained by adding 2 a 
total of k — 1 times to 1.) To show that Vk(P(k) — P(k + 1)) is true, we must show that if P(k) 
is true (the inductive hypothesis), then P(k + 1) is true. Note that P(k + 1) is the statement that 


14+345+---+(2k-1)+ Qk+1I) =&+1)?. 
So, assuming that P(k) is true, it follows that 


LS 3p Oh — 1) OR I=L 34--++ 2h D+ Ce 1) 


22+ (2k +1) 
=k? 4+2k+1 
=(k+1). 


This shows that P(k + 1) follows from P(k). Note that we used the inductive hypothesis P (k) 
in the second equality to replace the sum of the first k odd positive integers by k?. 


We have now completed both the basis step and the inductive step. That is, we have shown 
that P (1) is true and the conditional statement P(k) — P(k + 1) istrue for all positive integers k. 
Consequently, by the principle of mathematical induction we can conclude that P (7) is true for 
all positive integers n. That is, we know that 1 +3-+5-+---+(2n— 1) =n’ for all positive 
integers n. < 


Often, we will need to show that P(n) is true forn = b, b+ 1, b+ 2,..., where b is an 
integer other than 1. We can use mathematical induction to accomplish this, as long as we change 
the basis step by replacing P(1) with P(b). In other words, to use mathematical induction to 
show that P(n) is true forn = b,b+1,b+2,..., where b is an integer other than 1, we show 
that P(b) is true in the basis step. In the inductive step, we show that the conditional statement 
P(k) > P(k +1) is true fork = b,b+1,b+2,.... Note that b can be negative, zero, or 
positive. Following the domino analogy we used earlier, imagine that we begin by knocking 
down the bth domino (the basis step), and as each domino falls, it knocks down the next domino 
(the inductive step). We leave it to the reader to show that this form of induction is valid (see 
Exercise 83). 

We illustrate this notion in Example 3, which states that a summation formula is valid for 
all nonnegative integers. In this example, we need to prove that P() is true forn = 0,1, 2,.... 
So, the basis step in Example 3 shows that P (0) is true. 


318 5/ Induction and Recursion 


EXAMPLE 3 


EXAMPLE 4 


Use mathematical induction to show that 
for all nonnegative integers n. 


Solution: Let P(n) be the proposition that 1 + 2 + 2? +... +2" =2"+! — | for the integer n. 
BASIS STEP: P(0) is true because 2° = 1 = 2! — 1. This completes the basis step. 


INDUCTIVE STEP: For the inductive hypothesis, we assume that P(k) is true for an arbitrary 
nonnegative integer k. That is, we assume that 


es a ee ae Oe ee cee ee 


To carry out the inductive step using this assumption, we must show that when we assume 
that P(k) is true, then P(k + 1) is also true. That is, we must show that 


1 9" 4a ee agit! — 2k+)+1 —] — kt? _ 1 
assuming the inductive hypothesis P(k). Under the assumption of P(k), we see that 


TaD 2? eae PROP? a (1a DOP ines QF) et 


1H (Qk+1 _ 1) 4 okt 
=2.2k+1 —] 
= 2k+2 _ 1. 


Note that we used the inductive hypothesis in the second equation in this string of equalities to 
replace 1 + 2+ 27 +---+2* by 2‘+! — 1. We have completed the inductive step. 


Because we have completed the basis step and the inductive step, by mathematical induction 
we know that P(n) is true for all nonnegative integers n. That is, 1 + 2+---+2"=2"t!—] 
for all nonnegative integers n. < 


The formula given in Example 3 is a special case of a general result for the sum of terms 
of a geometric progression (Theorem | in Section 2.4). We will use mathematical induction to 
provide an alternative proof of this formula. 


Sums of Geometric Progressions Use mathematical induction to prove this formula for the 
sum of a finite number of terms of a geometric progression with initial term a and common 
ratio r: 


Me n+1 _ 

—_ 2 n_ a a 

ar’ =a+ar-+ar’+---+aro= 
j=0 


whenr # 1, 
r—1 


where v is a nonnegative integer. 


Solution: To prove this formula using mathematical induction, let P(n) be the statement that 
the sum of the first n + 1 terms of a geometric progression in this formula is correct. 


BASIS STEP: P(O) is true, because 


artl_qgoar—a_  a(r—1) 


a. 


r-l r-t r—1 
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INDUCTIVE STEP: The inductive hypothesis is the statement that P(k) is true, where k is an 
arbitrary nonnegative integer. That is, P(k) is the statement that 


get ag 


a+ar+ar*+---+ark= i 
r- 


To complete the inductive step we must show that if P(x) is true, then P(k + 1) is also true. To 
show that this is the case, we first add ar‘+! to both sides of the equality asserted by P(k). We 
find that 


k+1 

IH ar —a 

atar+ar*+---+ar® + ark! = 7 er 
r- 


Rewriting the right-hand side of this equation shows that 


are! a kt oe og ear 
—— +artl= + 
r—1 r—-1 r—-1 
ark+2 _ q 
r—1 
Combining these last two equations gives 
k+2 
: : ar“*“—a 
a+ar+ar*+---+ark + ar! = i 
r — 


This shows that if the inductive hypothesis P(x) is true, then P(k + 1) must also be true. This 
completes the inductive argument. 


We have completed the basis step and the inductive step, so by mathematical induction P (n) 
is true for all nonnegative integers n. This shows that the formula for the sum of the terms of a 
geometric series is correct. < 


As previously mentioned, the formula in Example 3 is the case of the formula in 
Example 4 with a = | and r = 2. The reader should verify that putting these values for a 
and r into the general formula gives the same formula as in Example 3. 


PROVING INEQUALITIES Mathematical induction can be used to prove a variety of 
inequalities that hold for all positive integers greater than a particular positive integer, as 
Examples 5-7 illustrate. 


Use mathematical induction to prove the inequality 
n <2" 
for all positive integers n. 


Solution: Let P(n) be the proposition that n < 2”. 
BASIS STEP: P(\) is true, because 1 < 2! = 2. This completes the basis step. 


INDUCTIVE STEP: We first assume the inductive hypothesis that P(k) is true for anarbitrary 
positive integer k. That is, the inductive hypothesis P(x) is the statement that k < 2‘.To complete 
the inductive step, we need to show that if P(x) is true, then P(k + 1), which is the statement 
that k + 1 < 2‘+!_ is true. That is, we need to show that if k < 2*, thenk + 1 < 2**+!. To show 


320 5/ Induction and Recursion 


EXAMPLE 6 


EXAMPLE 7 


that this conditional statement is true for the positive integer k, we first add 1 to both sides of 
k < 2*, and then note that 1 < 2*. This tells us that 


IH i 
cele ie a ae, 
This shows that P(k + 1) is true, namely, thatk + 1 < 2*+1 based on the assumption that P (k) 


is true. The induction step is complete. 


Therefore, because we have completed both the basis step and the inductive step, by 
the principle of mathematical induction we have shown that n < 2” is true for all positive 
integers n. 


Use mathematical induction to prove that 2” < n! for every integer n with n > 4. (Note that this 
inequality is false for n = 1, 2, and 3.) 

Solution: Let P(n) be the proposition that 2” < n!. 

BASIS STEP: To prove the inequality forn > 4 requires that the basis step be P(4). Note that 
P(4) is true, because 2* = 16 < 24 = 4! 

INDUCTIVE STEP: For the inductive step, we assume that P (xk) is true for an arbitrary integer k 
with k > 4. That is, we assume that 2* < k! for the positive integer k with k > 4. We must show 
that under this hypothesis, P(k + 1) is also true. That is, we must show that if 2k < k! for an 
arbitrary positive integer k where k > 4, then 2‘+! < (k + 1)!. We have 

g*tl — 9.9% by definition of exponent 

<2-k! by the inductive hypothesis 

<(kK+1)k! because2<k+1 

=(k+1)! _ by definition of factorial function. 
This shows that P(k + 1) is true when P(k) is true. This completes the inductive step of the 
proof. 

We have completed the basis step and the inductive step. Hence, by mathematical induction 
P(n) is true for all integers n with n > 4. That is, we have proved that 2” < n! is true for all 
integers n with n > 4. < 

An important inequality for the sum of the reciprocals of a set of positive integers will be 
proved in Example 7. 

An Inequality for Harmonic Numbers The harmonic numbers H;, j = 1,2,3,..., are 
defined by 

Ayj=1+ : + : a ada 

pees SOS a 
For instance, 

H. ee 

ah se ae ae 


Use mathematical induction to show that 
ae 
= 5 ; 


whenever n is a nonnegative integer. 
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Solution: To carry out the proof, let P(n) be the proposition that Hyn > 1 + - 


0 
BASIS STEP: P(0) is true, because H»o = Hy = 1>1+ 5 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(k) is true, that is, 


k 
Aye > 1+ rt where k is an arbitrary nonnegative integer. We must show that if P(x) is true, 


k+1 
then P(k + 1), which states that Hyi41 > 1+ = 
hypothesis, it follows that 


, is also true. So, assuming the inductive 


1 1 1 1 1 
Apes = 1+ 5 + 3 poset oT + kal pene RAT ee 
1 1 
= Ay + 4] ap eet aa sa oe of 2‘th harmonic 
>{1lt+ . + : + + : by the inductive hypothesi 
= soe the inducti othesis 
Ze 5 a] FI y the inductive hypothesi 
>{1+ 5 +2". SEAL because there are 2" terms 
each > 1/2k+1 
k 1 
>({l+=)+- canceling a common factor of 
2 2 i 
2" in second term 
k+1 
= 1+ ——_. 
- 2 


This establishes the inductive step of the proof. 


We have completed the basis step and the inductive step. Thus, by mathematical induction 
P(n) is true for all nonnegative integers n. That is, the inequality Hy. > 1 + o for the harmonic 
numbers holds for all nonnegative integers n. < 


Remark: The inequality established here shows that the harmonic series 


i. dl 
aap one 


1 
ToS 


1 
n 
is a divergent infinite series. This is an important example in the study of infinite series. 


PROVING DIVISIBILITY RESULTS Mathematical induction can be used to prove divisibil- 
ity results about integers. Although such results are often easier to prove using basic results in 
number theory, it is instructive to see how to prove such results using mathematical induction, 


as Examples 8 and 9 illustrate. oo ; : 
Use mathematical induction to prove that n> —n is divisible by 3 whenever n is a posi- 


tive integer. (Note that this is the statement with p = 3 of Fermat’s little theorem, which is 
Theorem 3 of Section 4.4.) 


3 


Solution: To construct the proof, let P(n) denote the proposition: “n° — n is divisible by 3.” 


BASIS STEP: The statement P(1) is true because 13 — 1 = 0 is divisible by 3. This completes 
the basis step. 


INDUCTIVE STEP: For the inductive hypothesis we assume that P(x) is true; that is, we 
assume that k? — k is divisible by 3 for an arbitrary positive integer k. To complete the inductive 
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step, we must show that when we assume the inductive hypothesis, it follows that P(k + 1), 
the statement that (k + 1)? — (k + 1) is divisible by 3, is also true. That is, we must show that 
(k + 1)? — (k + 1) is divisible by 3. Note that 


kt DP =F) HO 43 43k + C45 
= (RP —k)4+3(k2 +h). 


Using the inductive hypothesis, we conclude that the first term k* — k is divisible by 3. The 
second term is divisible by 3 because it is 3 times an integer. So, by part (i) of Theorem | in 
Section 4.1, we know that (k + 1)? — (k + 1) isalso divisible by 3. This completes the inductive 
step. 


Because we have completed both the basis step and the inductive step, by the prin- 
ciple of mathematical induction we know that n> —n is divisible by 3 whenever n is a 
positive integer. < 


The next example presents a more challenging proof by mathematical induction of a divis- 
ibility result. 


Use mathematical induction to prove that 7”+? + 82”+! is divisible by 57 for every nonnegative 
integer n. 


Solution: To construct the proof, let P(n) denote the proposition: “7”+? + 82”+! is divisible by 
S12 


BASIS STEP: To complete the basis step, we must show that P(0) is true, because we want 
to prove that P() is true for every nonnegative integer. We see that P(0) is true because 
79+? 4 g2-0+1 — 72 4 8! — 57 is divisible by 57. This completes the basis step. 


INDUCTIVE STEP: For the inductive hypothesis we assume that P(x) is true for an arbitrary 
nonnegative integer k; that is, we assume that 7+? + 874+! is divisible by 57. To complete the 
inductive step, we must show that when we assume that the inductive hypothesis P(k) is true, 
then P(k + 1), the statement that TARAI+2 4 Q2K+D+! ig divisible by 57, is also true. 

The difficult part of the proof is to see how to use the inductive hypothesis. To take advantage 
of the inductive hypothesis, we use these steps: 


TAFT 4 gt +L — 7k+3 4 g2k+3 
= 7. 7E+2 4 g2. g2ktl 
— 7. 7RH2 4 64. g2k41 
— 7 (72 4. g2kt1) 4.57. g2k+1, 


We can now use the inductive hypothesis, which states that 7+? + 874+! is divisible by 
57. We will use parts (1) and (ii) of Theorem | in Section 4.1. By part (ii) of this theorem, and 
the inductive hypothesis, we conclude that the first term in this last sum, qT + geet ly. is 
divisible by 57. By part (ii) of this theorem, the second term in this sum, 57 - 8°*+!, is divisible 
by 57. Hence, by part (i) of this theorem, we conclude that T(TRte 4 BH) 457. get! 
7k+3 4 92k+3 is divisible by 57. This completes the inductive step. 


Because we have completed both the basis step and the inductive step, by the principle of 
mathematical induction we know that 7”*? + 82”+! is divisible by 57 for every nonnegative 
integer n. 


PROVING RESULTS ABOUT SETS Mathematical induction can be used to prove many 
results about sets. In particular, in Example 10 we prove a formula for the number of subsets of 
a finite set and in Example 11 we establish a set identity. 


EXAMPLE 10 


EXAMPLE 11 
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FIGURE 3 Generating Subsets of a Set with k + 1 Elements. Here T = S U {a}. 


The Number of Subsets of a Finite Set Use mathematical induction to show that if S is a 
finite set with n elements, where n is a nonnegative integer, then S has 2” subsets. (We will 
prove this result directly in several ways in Chapter 6.) 


Solution: Let P(n) be the proposition that a set with n elements has 2” subsets. 


BASIS STEP: P(0) is true, because a set with zero elements, the empty set, has exactly 2° = 1 
subset, namely, itself. 


INDUCTIVE STEP: For the inductive hypothesis we assume that P(k) is true for an arbitrary 
nonnegative integer k, that is, we assume that every set with k elements has 2* subsets. It must 
be shown that under this assumption, P(k + 1), which is the statement that every set with k + 1 
elements has 2+! subsets, must also be true. To show this, let T be a set with k + 1 elements. 
Then, it is possible to write T = S U {a}, where a is one of the elements of T and S = T — {a} 
(and hence || = k). The subsets of T can be obtained in the following way. For each subset X 
of S there are exactly two subsets of T, namely, X and X U {a}. (This is illustrated in Figure 3.) 
These constitute all the subsets of T and are all distinct. We now use the inductive hypothesis 
to conclude that S has 2* subsets, because it has k elements. We also know that there are two 
subsets of T for each subset of S. Therefore, there are 2 - 2k — 2*+! subsets of T. This finishes 
the inductive argument. 


Because we have completed the basis step and the inductive step, by mathematical induction 
it follows that P(7) is true for all nonnegative integers n. That is, we have proved that a set 
with n elements has 2” subsets whenever n is a nonnegative integer. < 


Use mathematical induction to prove the following generalization of one of De Morgan’s laws: 


n n —_ 
(\4; = U 4 


j=l j=l 
whenever Aj, A2,..., Ay are subsets of a universal set U andn > 2. 


Solution: Let P(n) be the identity for n sets. 


BASIS STEP: The statement P(2) asserts that Ay N Az = A, U Ap. This is one of De Morgan’s 
laws; it was proved in Example 11 of Section 2.2. 
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EXAMPLE 12 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(k) is true, where k is an 
arbitrary integer with k > 2; that is, it is the statement that 


k 


k 
NAa= U4 
j= 


j=l 

whenever Aj, A2,..., Ax are subsets of the universal set U. To carry out the inductive step, 
we need to show that this assumption implies that P(k + 1) is true. That is, we need to show 
that if this equality holds for every collection of k subsets of U, then it must also hold for every 
collection of k + 1 subsets of U. Suppose that A, A2,..., Ag, Ag+1 are subsets of U. When 
the inductive hypothesis is assumed to hold, it follows that 


k+1 k 
() Aj () Aj N Ag+ by the definition of intersection 
j=l j=l 


U Axg41 by De Morgan’s law (where the two sets are ni — 1 Aj and Ag41) 


ll 
LE ) 
> 


U Ag41 by the inductive hypothesis 


II 
= 
>| 


k+l 
U Aj by the definition of union. 
j=l 


This completes the inductive step. 


Because we have completed both the basis step and the inductive step, by mathematical 
induction we know that P(n) is true whenever n is a positive integer, n > 2. That is, we know 
that 


n n = 
(4 = U4 


j=l j=l 
whenever A, A2,..., Ay are subsets of a universal set U andn > 2. < 


PROVING RESULTS ABOUT ALGORITHMS Next, we provide an example (somewhat 
more difficult than previous examples) that illustrates one of many ways mathematical induction 
is used in the study of algorithms. We will show how mathematical induction can be used to 
prove that a greedy algorithm we introduced in Section 3.1 always yields an optimal solution. 


Recall the algorithm for scheduling talks discussed in Example 7 of Section 3.1. The input to 
this algorithm is a group of m proposed talks with preset starting and ending times. The goal is 
to schedule as many of these lectures as possible in the main lecture hall so that no two talks 
overlap. Suppose that talk ¢; begins at time s; and ends at time e;. (No two lectures can proceed 
in the main lecture hall at the same time, but a lecture in this hall can begin at the same time 
another one ends.) 

Without loss of generality, we assume that the talks are listed in order of nondecreasing 
ending time, so that e; < ez <--- < e. The greedy algorithm proceeds by selecting at each 
stage a talk with the earliest ending time among all those talks that begin no sooner than when 


EXAMPLE 13 


Extra 
Examples 


< ) 


5.1 Mathematical Induction 325 


the last talk scheduled in the main lecture hall has ended. Note that a talk with the earliest end 
time is always selected first by the algorithm. We will show that this greedy algorithm is optimal 
in the sense that it always schedules the most talks possible in the main lecture hall. To prove 
the optimality of this algorithm we use mathematical induction on the variable n, the number 
of talks scheduled by the algorithm. We let P(n) be the proposition that if the greedy algorithm 
schedules n talks in the main lecture hall, then it is not possible to schedule more than n talks 
in this hall. 


BASIS STEP: Suppose that the greedy algorithm managed to schedule just one talk, 1, in 
the main lecture hall. This means that no other talk can start at or after e;, the end time of fy. 
Otherwise, the first such talk we come to as we go through the talks in order of nondecreasing 
end times could be added. Hence, at time e; each of the remaining talks needs to use the main 
lecture hall because they all start before e; and end after e;. It follows that no two talks can be 
scheduled because both need to use the main lecture hall at time e;. This shows that P(1) is 
true and completes the basis step. 


INDUCTIVE STEP: The inductive hypothesis is that P(k) is true, where k is an arbitrary 
positive integer, that is, that the greedy algorithm always schedules the most possible talks 
when it selects k talks, where k is a positive integer, given any set of talks, no matter how 
many. We must show that P(k + 1) follows from the assumption that P(x) is true, that is, we 
must show that under the assumption of P(k), the greedy algorithm always schedules the most 
possible talks when it selects k + | talks. 

Now suppose that the greedy algorithm has selected k + 1 talks. Our first step in completing 
the inductive step is to show there is a schedule including the most talks possible that contains 
talk t;, a talk with the earliest end time. This is easy to see because a schedule that begins with 
the talk ¢; in the list, where i > 1, can be changed so that talk ¢ replaces talk ¢;. To see this, note 
that because e; < e;, all talks that were scheduled to follow talk ¢; can still be scheduled. 

Once we included talk t;, scheduling the talks so that as many as possible are scheduled 
is reduced to scheduling as many talks as possible that begin at or after time e;. So, if we 
have scheduled as many talks as possible, the schedule of talks other than talk t; is an optimal 
schedule of the original talks that begin once talk t; has ended. Because the greedy algorithm 
schedules k talks when it creates this schedule, we can apply the inductive hypothesis to conclude 
that it has scheduled the most possible talks. It follows that the greedy algorithm has scheduled 
the most possible talks, k + 1, when it produced a schedule with k + 1 talks, so P(kK + 1) is 
true. This completes the inductive step. 


We have completed the basis step and the inductive step. So, by mathematical induction we 
know that P() is true for all positive integers n. This completes the proof of optimality. That is, 
we have proved that when the greedy algorithm schedules n talks, when n is a positive integer, 
then it is not possible to schedule more than n talks. 


CREATIVE USES OF MATHEMATICAL INDUCTION Mathematical induction can often 
be used in unexpected ways. We will illustrate two particularly clever uses of mathematical 
induction here, the first relating to survivors in a pie fight and the second relating to tilings with 
regular triominoes of checkerboards with one square missing. 


Odd Pie Fights An odd number of people stand in a yard at mutually distinct distances. 
At the same time each person throws a pie at their nearest neighbor, hitting this person. Use 
mathematical induction to show that there is at least one survivor, that is, at least one person 
who is not hit by a pie. (This problem was introduced by Carmony [Ca79]. Note that this result 
is false when there are an even number of people; see Exercise 75.) 


Solution: Let P(n) be the statement that there is a survivor whenever 2n + | people stand in 
a yard at distinct mutual distances and each person throws a pie at their nearest neighbor. To 
prove this result, we will show that P() is true for all positive integers n. This follows because 
as n runs through all positive integers, 2n + | runs through all odd integers greater than or equal 
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Links 


EXAMPLE 14 


FIGURE4 A 
Right Triomino. 


to 3. Note that one person cannot engage in a pie fight because there is no one else to throw the 
pie at. 


BASIS STEP: When n = 1, there are 2n + 1 = 3 people in the pie fight. Of the three people, 
suppose that the closest pair are A and B, and C is the third person. Because distances between 
pairs of people are different, the distance between A and C and the distance between B and C are 
both different from, and greater than, the distance between A and B. It follows that A and B throw 
pies at each other, while C throws apie at either A or B, whichever is closer. Hence, C is not hit by 
apie. This shows that at least one of the three people is not hit by a pie, completing the basis step. 


INDUCTIVE STEP: For the inductive step, assume that P(x) is true for an arbitrary odd 
integer k with k > 3. That is, assume that there is at least one survivor whenever 2k + | people 
stand in a yard at distinct mutual distances and each throws a pie at their nearest neighbor. We 
must show that if the inductive hypothesis P (x) is true, then P(k + 1), the statement that there is 
at least one survivor whenever 2(k + 1) + | = 2k + 3 people stand in a yard at distinct mutual 
distances and each throws a pie at their nearest neighbor, is also true. 

So suppose that we have 2(k + 1) + 1 = 2k +3 people in a yard with distinct distances 
between pairs of people. Let A and B be the closest pair of people in this group of 2k + 3 people. 
When each person throws a pie at the nearest person, A and B throw pies at each other. We have 
two cases to consider, (i) when someone else throws a pie at either A or B and (ii) when no one 
else throws a pie at either A or B. 


Case (i): Because A and B throw pies at each other and someone else throws a pie at either A 
and B, at least three pies are thrown at A and B, and at most (2k + 3) — 3 = 2k pies are thrown 
at the remaining 2k + | people. This guarantees that at least one person is a survivor, for if each 
of these 2k + | people was hit by at least one pie, a total of at least 2k + 1 pies would have to be 
thrown at them. (The reasoning used in this last step is an example of the pigeonhole principle 
discussed further in Section 6.2.) 


Case (ii): No one else throws a pie at either A and B. Besides A and B, there are 2k + 1 people. 
Because the distances between pairs of these people are all different, we can use the inductive 
hypothesis to conclude that there is at least one survivor S when these 2k + 1 people each 
throws a pie at their nearest neighbor. Furthermore, S is also not hit by either the pie thrown 
by A or the pie thrown by B because A and B throw their pies at each other, so S' is a survivor 
because S is not hit by any of the pies thrown by these 2k + 3 people. 


We have completed both the basis step and the inductive step, using a proof by cases. So by 
mathematical induction it follows that P (7) is true for all positive integers n. We conclude that 
whenever an odd number of people located in a yard at distinct mutual distances each throws a 
pie at their nearest neighbor, there is at least one survivor. < 


In Section 1.8 we discussed the tiling of checkerboards by polyominoes. Example 14 illus- 
trates how mathematical induction can be used to prove a result about covering checkerboards 
with right triominoes, pieces shaped like the letter “L.” 


Let n be a positive integer. Show that every 2” x 2” checkerboard with one square removed can 
be tiled using right triominoes, where these pieces cover three squares at a time, as shown in 
Figure 4. 


Solution: Let P (n) be the proposition that every 2” x 2” checkerboard with one square removed 
can be tiled using right triominoes. We can use mathematical induction to prove that P(n) is 
true for all positive integers n. 


BASIS STEP: P(1) is true, because each of the four 2 x 2 checkerboards with one square 
removed can be tiled using one right triomino, as shown in Figure 5. 


Consult Common Errors 
in Discrete Mathematics 
on this book’s website for 
more basic mistakes. 
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FIGURE 5 Tiling 2 x 2 Checkerboards with One Square Removed. 


INDUCTIVE STEP: The inductive hypothesis is the assumption that P (4) is true for the positive 
integer k; that is, it is the assumption that every 2" x 2* checkerboard with one square removed 
can be tiled using right triominoes. It must be shown that under the assumption of the inductive 
hypothesis, P(k + 1) must also be true; that is, any 2+! x 2+! checkerboard with one square 
removed can be tiled using right triominoes. 

To see this, consider a 2‘+! x 2'+! checkerboard with one square removed. Split this 
checkerboard into four checkerboards of size 2" x 2*, by dividing it in half in both directions. 
This is illustrated in Figure 6. No square has been removed from three of these four checker- 
boards. The fourth 2‘ x 2 checkerboard has one square removed, so we now use the inductive 
hypothesis to conclude that it can be covered by right triominoes. Now temporarily remove the 
square from each of the other three 2" x 2" checkerboards that has the center of the original, 
larger checkerboard as one of its corners, as shown in Figure 7. By the inductive hypothesis, 
each of these three 2‘ x 2 checkerboards with a square removed can be tiled by right triomi- 
noes. Furthermore, the three squares that were temporarily removed can be covered by one right 
triomino. Hence, the entire 2‘+! x 2+! checkerboard can be tiled with right triominoes. 


We have completed the basis step and the inductive step. Therefore, by mathemati- 
cal induction P(n) is true for all positive integers n. This shows that we can tile every 
2” x 2” checkerboard, where n is a positive integer, with one square removed, using right 
triominoes. 


FEF . 


FIGURE 6 _ Dividing a FIGURE7 Tiling the 
2*+1 x 2*+1 Checkerboard into 2k+1 x 2*+1 Checkerboard 
Four 2‘ x 2* Checkerboards. with One Square Removed. 


Mistaken Proofs By Mathematical Induction 


As with every proof method, there are many opportunities for making errors when using mathe- 
matical induction. Many well-known mistaken, and often entertaining, proofs by mathematical 
induction of clearly false statements have been devised, as exemplified by Example 15 and 
Exercises 49-51. Often, it is not easy to find where the error in reasoning occurs in such mis- 
taken proofs. 
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EXAMPLE 15 


To uncover errors in proofs by mathematical induction, remember that in every such proof, 
both the basis step and the inductive step must be done correctly. Not completing the basis step 
in a supposed proof by mathematical induction can lead to mistaken proofs of clearly ridiculous 
statements such as “‘n = n-+ | whenever n is a positive integer.” (We leave it to the reader to 
show that it is easy to construct a correct inductive step in an attempted proof of this statement.) 
Locating the error in a faulty proof by mathematical induction, as Example 15 illustrates, can 
be quite tricky, especially when the error is hidden in the basis step. 


Find the error in this “proof” of the clearly false claim that every set of lines in the plane, no 
two of which are parallel, meet in a common point. 


“Proof:” Let P(n) be the statement that every set of n lines in the plane, no two of which are 
parallel, meet in a common point. We will attempt to prove that P() is true for all positive 
integers n > 2. 


BASIS STEP: The statement P (2) is true because any two lines in the plane that are not parallel 
meet in a common point (by the definition of parallel lines). 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(x) is true for the positive 
integer k, that is, it is the assumption that every set of k lines in the plane, no two of which are 
parallel, meet in a common point. To complete the inductive step, we must show that if P(x) is 
true, then P(k + 1) must also be true. That is, we must show that if every set of k lines in the 
plane, no two of which are parallel, meet in a common point, then every set of k + | lines in the 
plane, no two of which are parallel, meet in a common point. So, consider a set of k + | distinct 
lines in the plane. By the inductive hypothesis, the first k of these lines meet in a common point 
P1- Moreover, by the inductive hypothesis, the last k of these lines meet in a common point p2. 
We will show that p; and p2 must be the same point. If p; and p2 were different points, all 
lines containing both of them must be the same line because two points determine a line. This 
contradicts our assumption that all these lines are distinct. Thus, p; and p2 are the same point. 
We conclude that the point py = p2 lies on all k+ 1 lines. We have shown that P(k + 1) is 
true assuming that P(x) is true. That is, we have shown that if we assume that every k, k > 2, 
distinct lines meet in a common point, then every k + 1 distinct lines meet in a common point. 
This completes the inductive step. 


We have completed the basis step and the inductive step, and supposedly we have a correct 
proof by mathematical induction. 


Solution: Examining this supposed proof by mathematical induction it appears that everything 
is in order. However, there is an error, as there must be. The error is rather subtle. Carefully 
looking at the inductive step shows that this step requires that k > 3. We cannot show that P(2) 
implies P(3). When k = 2, our goal is to show that every three distinct lines meet in a common 
point. The first two lines must meet in a common point p, and the last two lines must meet in 
a common point p2. But in this case, p; and p2 do not have to be the same, because only the 
second line is common to both sets of lines. Here is where the inductive step fails. < 


Guidelines for Proofs by Mathematical Induction 


Examples 1-14 illustrate proofs by mathematical induction of a diverse collection of theorems. 
Each of these examples includes all the elements needed in a proof by mathematical induction. 
We have provided an example of an invalid proof by mathematical induction. Summarizing what 
we have learned from these examples, we can provide some useful guidelines for constructing 
correct proofs by mathematical induction. We now present these guidelines. 
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Template for Proofs by Mathematical Induction 


Il 
2 


3. 
. State, and clearly identify, the inductive hypothesis, in the form “‘assume that P(k) is true 


Express the statement that is to be proved in the form “for all n > b, P(n)” for a fixed 
integer b. 

Write out the words “Basis Step.” Then show that P(b) is true, taking care that the correct 
value of b is used. This completes the first part of the proof. 

Write out the words “Inductive Step.” 


for an arbitrary fixed integer k > b.” 


. State what needs to be proved under the assumption that the inductive hypothesis is true. 


That is, write out what P(k + 1) says. 


. Prove the statement P(k + 1) making use the assumption P(k). Be sure that your proof 


is valid for all integers k with k > b, taking care that the proof works for small values 
of k, including k = b. 


. Clearly identify the conclusion of the inductive step, such as by saying “this completes 


the inductive step.” 


. After completing the basis step and the inductive step, state the conclusion, namely that 


by mathematical induction, P(n) is true for all integers n with n > b. 


It is worthwhile to revisit each of the mathematical induction proofs in Examples 1-14 to see 


how these steps are completed. It will be helpful to follow these guidelines in the solutions of the 
exercises that ask for proofs by mathematical induction. The guidelines that we presented can 
be adapted for each of the variants of mathematical induction that we introduce in the exercises 
and later in this chapter. 


Exercises 


. There are infinitely many stations on a train route. Sup- 
pose that the train stops at the first station and suppose 
that if the train stops at a station, then it stops at the next 
station. Show that the train stops at all stations. 


. Suppose that you know that a golfer plays the first hole of 
a golf course with an infinite number of holes and that if 
this golfer plays one hole, then the golfer goes on to play 
the next hole. Prove that this golfer plays every hole on 
the course. 


. Let P(n) be the statement that 137+ 23+... 


f) Explain why these steps show that this formula is true 
whenever 7 is a positive integer. 

+n = 

(n(n + 1)/2)? for the positive integer n. 

a) What is the statement P(1)? 


b) Show that P(1) is true, completing the basis step of 
the proof. 


c) What is the inductive hypothesis? 


d) What do you need to prove in the inductive step? 


Use mathematical induction in Exercises 3-17 to prove sum- 


mation formulae. Be sure to identify where you use the in- e) Complete the inductive step, identifying where you 


ductive hypothesis. 


3. Let P(n) be the statement that 174+ 27+.--+n? = 


n(n + 1)(2n + 1)/6 for the positive integer n. 
a) What is the statement P(1)? 


b) Show that P(1) is true, completing the basis step of 
the proof. 


c) What is the inductive hypothesis? 
d) What do you need to prove in the inductive step? 


e) Complete the inductive step, identifying where you 
use the inductive hypothesis. 


use the inductive hypothesis. 


f) Explain why these steps show that this formula is true 
whenever 7 is a positive integer. 


. Prove that 174+3?+5?+---4+(2n4+1)%? =(n4+1) 


(2n + 1)(2n + 3)/3 whenever n is a nonnegative integer. 


. Prove that 1-1!-+2-2!+---+n-n!=(n+)!-1 


whenever 7 is a positive integer. 


. Prove that3 ++3-5+3-52-+----+3-5"=3(5"+1_ 1)/4 


whenever 7 is a nonnegative integer. 


. Prove that 2—2-742-7—---+2(-7" =(1— 


(—7)"+!)/4 whenever n is a nonnegative integer. 
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9. 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


a) Find a formula for the sum of the first n even positive 
integers. 
b) Prove the formula that you conjectured in part (a). 


a) Find a formula for 
1 1 1 
by examining the values of this expression for small 


values of n. 
b) Prove the formula you conjectured in part (a). 


a) Find a formula for 
! + : + ; ea : 
2 4 8 2h 


by examining the values of this expression for small 
values of n. 
b) Prove the formula you conjectured in part (a). 


Prove that 


n ( yy = re 
eS yy haa 3.2" 


j=0 


whenever n is a nonnegative integer. 
Prove that 17 — 27 + 3? —.-. + (-1)""!n? = (-1)""! 
n(n + 1)/2 whenever n is a positive integer. 


Prove that for every positive integer n, baer os = 
(n — 1)2"*1 4.2. 
Prove that for every positive integer n, 


1-242-34---+n(n+1) =ntnt+ Ym 2)/3. 


Prove that for every positive integer n, 
1-2-34+2-3-4+---+n(n+ 1)(n+ 2) 
=n(n+ 1)(n+2)(n + 3)/4. 


Prove that )""_ j4 =n(n+ 1)(2n + 1) Gn? + 3n —1)/30 
whenever 7 is a positive integer. 


Use mathematical induction to prove the inequalities in Exer- 
cises 18-30. 


18. 


19. 


Let P(n) be the statement that n! < n”, where n is an 

integer greater than 1. 

a) What is the statement P(2)? 

b) Show that P(2) is true, completing the basis step of 
the proof. 

c) What is the inductive hypothesis? 

d) What do you need to prove in the inductive step? 

e) Complete the inductive step. 

f) Explain why these steps show that this inequality is 
true whenever n is an integer greater than 1. 


Let P(n) be the statement that 
1+ : + ; eae ! 2 : 
= —, eae — < —_ — a 
4 9 n2 n 


where n is an integer greater than 1. 

a) What is the statement P(2)? 

b) Show that P(2) is true, completing the basis step of 
the proof. 


20. 
21. 
22. 
23. 
24. 


* 25. 


* 26. 


* 27, 


28. 


c) What is the inductive hypothesis? 

d) What do you need to prove in the inductive step? 

e) Complete the inductive step. 

f) Explain why these steps show that this inequality is 
true whenever n is an integer greater than 1. 

Prove that 3” < n! if n is an integer greater than 6. 


Prove that 2” > n? if n is an integer greater than 4. 


For which nonnegative integers n is n2 


answer. 


< n!? Prove your 


For which nonnegative integers n is 2n + 3 < 2”? Prove 
your answer. 


Prove that 1/(2n) <[1-3-5----- (2n — 1)]/(2-4- 
--++2n) whenever n is a positive integer. 
Prove thatifh > —1,then1 + nh < (1 +h)” forall non- 


negative integers n. This is called Bernoulli’s inequality. 


Suppose that a and b are real numbers with 0 < b <a. 
Prove that if n is a positive integer, then a” — b” < 
na"—'(a —b). 


Prove that for every positive integer n, 


1 1 1 
1+ — 4+ —=4.---+ = 5 2Vn+1-D. 
tat a? sare ie (Jn + ) 


Prove that n? — 7n + 12 is nonnegative whenever n is an 
integer with n > 3. 


In Exercises 29 and 30, H,, denotes the nth harmonic number. 


* 29, 


* 30. 


Prove that Hy» < 1 +n whenever n is a nonnegative in- 
teger. 


Prove that 


A,+ AWo+---+H, =(n+1)A, —n. 


Use mathematical induction in Exercises 31—37 to prove di- 
visibility facts. 


31. 


32. 


33. 


34. 


* 35. 


* 36. 


* 37, 


Prove that 2 divides n? +n whenever n is a positive in- 
teger. 

Prove that 3 divides n>? + 2n whenever n is a positive 
integer. 

Prove that 5 divides n> — n whenever n is a nonnegative 
integer. 

Prove that 6 divides n? — n whenever n is a nonnegative 
integer. 

Prove that n? — 1 is divisible by 8 whenever n is an odd 
positive integer. 

Prove that 21 divides 4”+! + 52”! whenever n is a pos- 
itive integer. 

Prove that if n is a positive integer, then 133 divides 
qt! ne 122”-1. 


Use mathematical induction in Exercises 38-46 to prove re- 
sults about sets. 


38. 


Prove that if A,, A2,..., A, and B,, Bo,..., By, are sets 
such that A; © B; for j = 1,2,...,n, then 

n n 

U Aj Cc U B; 


39. 


40. 


41. 


42. 


43. 


44, 


45. 


* 46. 


Prove that if A,;, A2,..., A, and B,, Bo,..., By, are sets 
such that A; C B; for j = 1,2,...,n, then 


n n 
(ane () 8). 
j=l j=l 
Prove that if Aj, A2,..., A, and B are sets, then 


(Ay, N A2N---N Ay) UB 


= (A, UB)N(A2U B)N ++: (Ay U B). 


Prove that if Aj, A2,..., A, and B are sets, then 
(Aj UA2U---UA,) NB 


= (A, N B)U(A2N B)U---U (Ay NB). 


Prove that if Aj, A2,..., A, and B are sets, then 
(A; — B)N (A2 — B)N---N (Ay — B) 
= (A; NA2N-:-N Ay) — B. 


Prove that if A,, A2,.. 
set U, then 


., Ay, are subsets of a universal 


n 
LU Ak =(\e=1 Ak. 
k=1 


Prove that if Aj, A2,..., A, and B are sets, then 


(Ay — B)U (A) — B)U-+-U (Ay — B) 
= (Aj UA2U---UA,) — B. 


Prove that a set with n elements has n(n — 1)/2 subsets 
containing exactly two elements whenever n is an integer 
greater than or equal to 2. 


Prove that a set with n elements has n(n — 1)(n — 2)/6 
subsets containing exactly three elements whenever n is 
an integer greater than or equal to 3. 


In Exercises 47 and 48 we consider the problem of placing 
towers along a straight road, so that every building on the 
road receives cellular service. Assume that a building receives 
cellular service if it is within one mile of a tower. 


47. 


* 48. 


Devise a greedy algorithm that uses the minimum number 
of towers possible to provide cell service to d buildings 
located at positions x}, x2,...,Xq from the start of the 
road. [Hint: At each step, go as far as possible along the 
road before adding a tower so as not to leave any buildings 
without coverage. ] 

Use mathematical induction to prove that the algorithm 
you devised in Exercise 47 produces an optimal solution, 
that is, that it uses the fewest towers possible to provide 
cellular service to all buildings. 


Exercises 49-51 present incorrect proofs using mathemati- 
cal induction. You will need to identify an error in reasoning 
in each exercise. 


49. 


What is wrong with this “proof” that all horses are the 
same color? 

Let P(n) be the proposition that all the horses in a set 
of n horses are the same color. 


Basis Step: Clearly, P (1) is true. 


50. 


51. 


52. 


* 53. 


54. 


*55, 
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Inductive Step: Assume that P(k) is true, so that all 
the horses in any set of k horses are the same color. 
Consider any k +1 horses; number these as horses 
1,2,3,...,k,k +1. Now the first k of these horses all 
must have the same color, and the last k of these must 
also have the same color. Because the set of the first k 
horses and the set of the last k horses overlap, all k + 1 
must be the same color. This shows that P(k + 1) is true 
and finishes the proof by induction. 

What is wrong with this “proof”? 

“Theorem” For every positive integer n, )°/_,i = 
(n+ 5)?/2. 

Basis Step: The formula is true for n = 1. 


Inductive Step: Suppose that )*_,i = (n+ 5)?/2. 
Then pee, i=QOL,)+(+1). By the induc- 
tive hypothesis, ris (nt4y/2+n4+1= 
(tnth)/2+n4+1 = (? + 3n + 9/2 = 
(n + 3)°/2 = [(n + 1) + 31?/2, completing the induc- 
tive step. 

What is wrong with this “proof”? 

“Theorem” For every positive integer n, if x and y are 
positive integers with max(x, y) =n, then x = y. 


Basis Step: Suppose that n = 1. If max(x, y) = 1 and x 
and y are positive integers, we have x = | and y = 1. 


Inductive Step: Let k be a positive integer. Assume that 
whenever max(x, y) = k and x and y are positive inte- 
gers, then x = y. Now let max(x, y) =k +1, where x 
and y are positive integers. Then max(x — 1, y— 1) =k, 
so by the inductive hypothesis, x — 1 = y — 1. It follows 
that x = y, completing the inductive step. 

Suppose that m and n are positive integers with m > n 
and f is a function from {1,2,...,m} to {1,2,...,n}. 
Use mathematical induction on the variable n to show 
that f is not one-to-one. 

Use mathematical induction to show that n people can di- 
vide a cake (where each person gets one or more separate 
pieces of the cake) so that the cake is divided fairly, that 
is, in the sense that each person thinks he or she got at 
least (1/n)th of the cake. [Hint: For the inductive step, 
take a fair division of the cake among the first k people, 
have each person divide their share into what this per- 
son thinks are k + 1 equal portions, and then have the 
(kK + 1)st person select a portion from each of the k peo- 
ple. When showing this produces a fair division for k + 1 
people, suppose that person k + 1 thinks that personi got 
p; of the cake where ea Pi = 1.) 

Use mathematical induction to show that given a set of 
n+ 1 positive integers, none exceeding 2n, there is at 
least one integer in this set that divides another integer in 
the set. 

A knight on a chessboard can move one space horizon- 
tally (in either direction) and two spaces vertically (in 
either direction) or two spaces horizontally (in either di- 
rection) and one space vertically (in either direction). 
Suppose that we have an infinite chessboard, made up 
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56. 


57. 


58. 


59. 


60. 


*61. 


* 62. 


** 63, 


64. 


65. 


of all squares (m,n) where m and n are nonnegative inte- 
gers that denote the row number and the column number 
of the square, respectively. Use mathematical induction to 
show that a knight starting at (0, 0) can visit every square 
using a finite sequence of moves. [Hint: Use induction 
on the variable s = m + n.] 


Suppose that 


a 0 
a=[5 of 


where a and b are real numbers. Show that 

Ane a” 0 

0 b 

for every positive integer n. 
(Requires calculus) Use mathematical induction to prove 
that the derivative of f(x) = x” equals nx”—! whenever 
n is a positive integer. (For the inductive step, use the 
product rule for derivatives.) 
Suppose that A and B are square matrices with the prop- 
erty AB = BA. Show that AB” = B”A for every positive 
integer n. 
Suppose that m is a positive integer. Use mathematical 
induction to prove that if a and b are integers witha = b 
(mod m), then ak = bk (mod m) whenever k is a nonneg- 
ative integer. 
Use mathematical induction to show that =(p; V p2 V 
-++V pn) iS equivalent to =p; A7p2 A ---A=pn 
whenever p1, P2,.--, Pn are propositions. 
Show that 


[(p1 > p2) A (p2 > p3) A+++ A (Pn-1 > Pn] 
= ((p1 A pr2A-++ A Pn-1) —> Pn) 


is a tautology whenever p1, p2,... 
where n > 2. 


, Pn are propositions, 


Show that lines separate the plane into (n? +n + 2)/2 
regions if no two of these lines are parallel and no three 
pass through a common point. 


Let aj, a2,..., dn be positive real numbers. The arith- 
metic mean of these numbers is defined by 


A= (aq, +42 +-+-+4n)/n, 


and the geometric mean of these numbers is defined by 


G = (ajan-+-dy)'/". 


Use mathematical induction to prove that A > G. 


Use mathematical induction to prove Lemma 3 of 
Section 4.3, which states that if p is a prime 
and p|dja2---d,, where a; is an integer for 
i=1,2,3,...,n, then p | a; for some integer 7. 


Show that if n is a positive integer, then 


> 
a\a2-:-:ar 


{a1,...,an}E{1,2,...,0} 


=Nn. 


(Here the sum is over all nonempty subsets of the set of 
the n smallest positive integers.) 


* 66. 


67. 


* 68. 


Use the well-ordering property to show that the follow- 
ing form of mathematical induction is a valid method to 
prove that P (7) is true for all positive integers n. 


Basis Step: P(1) and P(2) are true. 


Inductive Step: For each positive integer k, if P(k) and 
P(k + 1) are both true, then P(k + 2) is true. 


Show that if Aj, A2,..., A, are sets where n > 2, and 
for all pairs of integers i and j with | <i < j < neither 
A; is a subset of A; or A; is a subset of A;, then there is 
an integer i, | <i <n such that A; is a subset of A; for 
all integers j with 1 < j <n. 


A guest at a party is a celebrity if this person is known 
by every other guest, but knows none of them. There is at 
most one celebrity at a party, for if there were two, they 
would know each other. A particular party may have no 
celebrity. Your assignment is to find the celebrity, if one 
exists, at a party, by asking only one type of question— 
asking a guest whether they know a second guest. Ev- 
eryone must answer your questions truthfully. That is, if 
Alice and Bob are two people at the party, you can ask Al- 
ice whether she knows Bob; she must answer correctly. 
Use mathematical induction to show that if there are n 
people at the party, then you can find the celebrity, if 
there is one, with 3(n — 1) questions. [Hint: First ask a 
question to eliminate one person as a celebrity. Then use 
the inductive hypothesis to identify a potential celebrity. 
Finally, ask two more questions to determine whether that 
person is actually a celebrity. ] 


Suppose there are n people in a group, each aware of a scandal 
no one else in the group knows about. These people commu- 
nicate by telephone; when two people in the group talk, they 
share information about all scandals each knows about. For 
example, on the first call, two people share information, so 
by the end of the call, each of these people knows about two 
scandals. The gossip problem asks for G(n), the minimum 
number of telephone calls that are needed for all n people to 
learn about all the scandals. Exercises 69-71 deal with the 
gossip problem. 


69. 
70. 


** 71, 
* 72% 


* 73. 


Find G(1), G(2), G(3), and G(4). 


Use mathematical induction to prove that G(n) < 2n — 4 
for n > 4. [Hint: In the inductive step, have a new person 
call a particular person at the start and at the end.] 


Prove that G(n) = 2n — 4 forn > 4. 


Show that itis possible to arrange the numbers 1, 2,...,” 
in a row so that the average of any two of these numbers 
never appears between them. [Hint: Show that it suffices 
to prove this fact when n is a power of 2. Then use math- 
ematical induction to prove the result when n is a power 
of 2.] 


Show that if 11, o,..., I, is a collection of open in- 
tervals on the real number line, n > 2, and every pair 
of these intervals has a nonempty intersection, that is, 
TI; 01; A@ whenever 1 <i <n and 1 < j <n, then 
the intersection of all these sets is nonempty, that is, 
AhA-:--AI, €@. (Recall that an open interval is 


the set of real numbers x witha < x < b, where a and b 
are real numbers witha < b.) 


Sometimes we cannot use mathematical induction to prove 
a result we believe to be true, but we can use mathematical 
induction to prove a stronger result. Because the inductive hy- 
pothesis of the stronger result provides more to work with, this 
process is called inductive loading. We use inductive loading 
in Exercise 74. 


74. Suppose that we want to prove that 


76. 


77. 


78. 
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person throws a pie at their nearest neighbor, it is possible 
that everyone is hit by a pie. 

Construct a tiling using right triominoes of the 4 x 4 
checkerboard with the square in the upper left corner re- 
moved. 

Construct a tiling using right triominoes of the 8 x 8 
checkerboard with the square in the upper left corner re- 
moved. 

Prove or disprove that all checkerboards of these shapes 
can be completely covered using right triominoes when- 
ever 71 is a positive integer. 


1 3 aed 2 ao nee 
Pere ah 32 ss 
ah eee ox" d) 6" x6" 
for all positive integers n. *79, Show that a three-dimensional 2” x 2” x 2” checker- 
a) Show that if we try to prove this inequality using math- board with one 1 x 1 x | cube missing can be completely 
ematical induction, the basis step works, but the in- covered by 2 x 2 x 2 cubes with one 1 x 1 x 1 cube re- 
ductive step fails. moved. 
* 80. Show that an n x n checkerboard with one square re- 
b) Show that mathematical induction can be used to moved can be completely covered using right triominoes 
prove the stronger inequality ifn > 5, nis odd, and3 Jn. 
1 3 2n—1 1 81. Show that a5 x 5 checkerboard with a corner square re- 
a aaa? < Van +1 acovet can be tiled using et triominoes. 
* 82. Find a 5 x 5 checkerboard with a square removed that 
for all integers greater than 1, which, together with a cannot be tiled using right triominoes. Prove that such a 
verification for the case where n = 1, establishes the tiling does not exist for this board. 
weaker inequality we originally tried to prove using 83. Use the principle of mathematical induction to show that 


mathematical induction. 


75. Let n be an even positive integer. Show that when n peo- 
ple stand in a yard at mutually distinct distances and each 


P(n) is true forn = b,b+1,b+2,..., where b is an 
integer, if P(b) is true and the conditional statement 
P(k) > P(k + 1) is true for all integers k with k > b. 


Strong Induction and Well-Ordering 


Introduction 


In Section 5.1 we introduced mathematical induction and we showed how to use it to prove a 
variety of theorems. In this section we will introduce another form of mathematical induction, 
called strong induction, which can often be used when we cannot easily prove a result using 
mathematical induction. The basis step of a proof by strong induction is the same as a proof of 
the same result using mathematical induction. That is, in a strong induction proof that P(7) is 
true for all positive integers n, the basis step shows that P (1) is true. However, the inductive steps 
in these two proof methods are different. In a proof by mathematical induction, the inductive 
step shows that if the inductive hypothesis P(k) is true, then P(k + 1) is also true. In a proof 
by strong induction, the inductive step shows that if P(j) is true for all positive integers not 
exceeding k, then P(k + 1) is true. That is, for the inductive hypothesis we assume that P(/) 
is true for 7 = 1,2,...,k. 

The validity of both mathematical induction and strong induction follow from the well- 
ordering property in Appendix 1. In fact, mathematical induction, strong induction, and well- 
ordering are all equivalent principles (as shown in Exercises 41, 42, and 43). That is, the validity 
of each can be proved from either of the other two. This means that a proof using one of these 
two principles can be rewritten as a proof using either of the other two principles. Just as it is 
sometimes the case that it is much easier to see how to prove a result using strong induction 
rather than mathematical induction, it is sometimes easier to use well-ordering than one of the 
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two forms of mathematical induction. In this section we will give some examples of how the 
well-ordering property can be used to prove theorems. 


Strong Induction 


Before we illustrate how to use strong induction, we state this principle again. 


STRONG INDUCTION To prove that P (7) is true for all positive integers n, where P(n) 
is a propositional function, we complete two steps: 


BASIS STEP: We verify that the proposition P (1) is true. 


INDUCTIVE STEP: We show that the conditional statement [P(1) A P(2)A---A 
P(k)] — P(k + 1) is true for all positive integers k. 


Note that when we use strong induction to prove that P(7) is true for all positive integers n, 
our inductive hypothesis is the assumption that P(/) is true for 7 = 1,2,...,k. That is, the 
inductive hypothesis includes all k statements P(1), P(2),..., P(k). Because we can use all k 
statements P(1), P(2),..., P(k) to prove P(k + 1), rather than just the statement P(k) as ina 
proof by mathematical induction, strong induction is a more flexible proof technique. Because 
of this, some mathematicians prefer to always use strong induction instead of mathematical 
induction, even when a proof by mathematical induction is easy to find. 

You may be surprised that mathematical induction and strong induction are equivalent. 
That is, each can be shown to be a valid proof technique assuming that the other is valid. In 
particular, any proof using mathematical induction can also be considered to be a proof by strong 
induction because the inductive hypothesis of a proof by mathematical induction is part of the 
inductive hypothesis in a proof by strong induction. That is, if we can complete the inductive 
step of a proof using mathematical induction by showing that P(k + 1) follows from P(k) for 
every positive integer k, then it also follows that P(k + 1) follows from all the statements P (1), 
P(2),..., P(k), because we are assuming that not only P(k) is true, but also more, namely, that 
the k — 1 statements P(1), P(2),..., P(k — 1) are true. However, it is much more awkward to 
convert a proof by strong induction into a proof using the principle of mathematical induction. 
(See Exercise 42.) 

Strong induction is sometimes called the second principle of mathematical induction 
or complete induction. When the terminology “complete induction” is used, the principle of 
mathematical induction is called incomplete induction, a technical term that is a somewhat 
unfortunate choice because there is nothing incomplete about the principle of mathematical 
induction; after all, it is a valid proof technique. 


STRONG INDUCTION AND THE INFINITE LADDER To better understand strong in- 
duction, consider the infinite ladder in Section 5.1. Strong induction tells us that we can reach 
all rungs if 


1. we can reach the first rung, and 
2. for every integer k, if we can reach all the first k rungs, then we can reach the (k + 1)st rung. 


That is, if P (7) is the statement that we can reach the nth rung of the ladder, by strong induction 
we know that P (n) is true for all positive integers n, because (1) tells us P(1) is true, completing 
the basis step and (2) tells us that P(1) A P(2) A--- A P(k) implies P(k + 1), completing the 
inductive step. 

Example 1 illustrates how strong induction can help us prove a result that cannot easily be 
proved using the principle of mathematical induction. 


EXAMPLE 1 
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Suppose we can reach the first and second rungs of an infinite ladder, and we know that if we 
can reach a rung, then we can reach two rungs higher. Can we prove that we can reach every 
rung using the principle of mathematical induction? Can we prove that we can reach every rung 
using strong induction? 


Solution: We first try to prove this result using the principle of mathematical induction. 


BASIS STEP: The basis step of such a proof holds; here it simply verifies that we can reach 
the first rung. 


ATTEMPTED INDUCTIVE STEP: The inductive hypothesis is the statement that we can 
reach the kth rung of the ladder. To complete the inductive step, we need to show that if we 
assume the inductive hypothesis for the positive integer k, namely, if we assume that we can 
reach the kth rung of the ladder, then we can show that we can reach the (k + 1)st rung of the 
ladder. However, there is no obvious way to complete this inductive step because we do not 
know from the given information that we can reach the (A + 1)st rung from the kth rung. After 
all, we only know that if we can reach a rung we can reach the rung two higher. 


Now consider a proof using strong induction. 


BASIS STEP: The basis step is the same as before; it simply verifies that we can reach the first 
rung. 


INDUCTIVE STEP: The inductive hypothesis states that we can reach each of the first k rungs. 
To complete the inductive step, we need to show that if we assume that the inductive hypothesis 
is true, that is, if we can reach each of the first k rungs, then we can reach the (k + 1)st rung. 
We already know that we can reach the second rung. We can complete the inductive step by 
noting that as long as k > 2, we can reach the (k + 1)st rung from the (k — 1)st rung because 
we know we can climb two rungs from a rung we can already reach, and because k — 1 < k, 
by the inductive hypothesis we can reach the (k — 1)st rung. This completes the inductive step 
and finishes the proof by strong induction. 


We have proved that if we can reach the first two rungs of an infinite ladder and for every 
positive integer k if we can reach all the first k rungs then we can reach the (k + 1)st rung, then 
we can reach all rungs of the ladder. 


Examples of Proofs Using Strong Induction 


Now that we have both mathematical induction and strong induction, how do we decide which 
method to apply in a particular situation? Although there is no cut-and-dried answer, we can 
supply some useful pointers. In practice, you should use mathematical induction when it is 
straightforward to prove that P(k) — P(k + 1) is true for all positive integers k. This is the 
case for all the proofs in the examples in Section 5.1. In general, you should restrict your use of 
the principle of mathematical induction to such scenarios. Unless you can clearly see that the 
inductive step of a proof by mathematical induction goes through, you should attempt a proof 
by strong induction. That is, use strong induction and not mathematical induction when you 
see how to prove that P(k + 1) is true from the assumption that P(/) is true for all positive 
integers j not exceeding k, but you cannot see how to prove that P(k + 1) follows from just 
P(k). Keep this in mind as you examine the proofs in this section. For each of these proofs, 
consider why strong induction works better than mathematical induction. 

We will illustrate how strong induction is employed in Examples 2-4. In these examples, 
we will prove a diverse collection of results. Pay particular attention to the inductive step in 
each of these examples, where we show that a result P(k + 1) follows under the assumption that 
P(j) holds for all positive integers 7 not exceeding k, where P(n) is a propositional function. 
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EXAMPLE 2 


Extra p> 
Examples 2 


EXAMPLE 3 


We begin with one of the most prominent uses of strong induction, the part of the fundamental 
theorem of arithmetic that tells us that every positive integer can be written as the product of 
primes. 


Show that if 7 is an integer greater than 1, then n can be written as the product of primes. 


Solution: Let P(n) be the proposition that n can be written as the product of primes. 


BASIS STEP: P(2) is true, because 2 can be written as the product of one prime, itself. (Note 
that P(2) is the first case we need to establish.) 


INDUCTIVE STEP: The inductive hypothesis is the assumption that P(j) is true for all 
integers j with 2 < j < k, that is, the assumption that j can be written as the product of primes 
whenever j is a positive integer at least 2 and not exceeding k. To complete the inductive step, 
it must be shown that P(k + 1) is true under this assumption, that is, that k + 1 is the product 
of primes. 

There are two cases to consider, namely, when k + | is prime and when k + | is composite. 
If k + | is prime, we immediately see that P(k + 1) is true. Otherwise, k + 1 is composite and 
can be written as the product of two positive integers a and b with2 <a <b <k + 1. Because 
both a and b are integers at least 2 and not exceeding k, we can use the inductive hypothesis to 
write both a and b as the product of primes. Thus, if k + 1 is composite, it can be written as the 
product of primes, namely, those primes in the factorization of a and those in the factorization 
of b. < 


Remark: Because | can be thought of as the empty product of no primes, we could have started 
the proof in Example 2 with P(1) as the basis step. We chose not to do so because many people 
find this confusing. 


Example 2 completes the proof of the fundamental theorem of arithmetic, which asserts that 
every nonnegative integer can be written uniquely as the product of primes in nondecreasing 
order. We showed in Section 4.3 that an integer has at most one such factorization into primes. 
Example 2 shows there is at least one such factorization. 

Next, we show how strong induction can be used to prove that a player has a winning 
strategy in a game. 


Consider a game in which two players take turns removing any positive number of matches they 
want from one of two piles of matches. The player who removes the last match wins the game. 
Show that if the two piles contain the same number of matches initially, the second player can 
always guarantee a win. 


Solution: Let n be the number of matches in each pile. We will use strong induction to prove 
P(n), the statement that the second player can win when there are initially n matches in each 
pile. 


BASIS STEP: Whenn = 1, the first player has only one choice, removing one match from one 
of the piles, leaving a single pile with a single match, which the second player can remove to 
win the game. 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(j) is true for all 7 with 
1 < j <k, that is, the assumption that the second player can always win whenever there are j 
matches, where | < j < k ineach of the two piles at the start of the game. We need to show that 
P(k + 1) is true, that is, that the second player can win when there are initially k + | matches in 
each pile, under the assumption that P(/) is true for 7 = 1, 2,..., k. So suppose that there are 
k + 1 matches in each of the two piles at the start of the game and suppose that the first player 
removes r matches (1 < r < k) from one of the piles, leaving & + 1 — r matches in this pile. 
By removing the same number of matches from the other pile, the second player creates the 


EXAMPLE 4 
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situation where there are two piles each with k + 1 —r matches. Because | <k+1-—-—r <k, 
we can now use the inductive hypothesis to conclude that the second player can always win. 
We complete the proof by noting that if the first player removes all k + 1 matches from one of 
the piles, the second player can win by removing all the remaining matches. 


Using the principle of mathematical induction, instead of strong induction, to prove the 
results in Examples 2 and 3 is difficult. However, as Example 4 shows, some results can be 
readily proved using either the principle of mathematical induction or strong induction. 

Before we present Example 4, note that we can slightly modify strong induction to handle 
a wider variety of situations. In particular, we can adapt strong induction to handle cases where 
the inductive step is valid only for integers greater than a particular integer. Let b be a fixed 
integer and j a fixed positive integer. The form of strong induction we need tells us that P(n) 
is true for all integers n with n > b if we can complete these two steps: 


BASIS STEP: We verify that the propositions P(b), P(b + 1),..., P(b + j) are true. 


INDUCTIVE STEP: We show that [P(b) A P(b+ 1) A--+A P(k)] ~ P(k + 1) is true for 
every integer k > b+ j. 


We will use this alternative form in the strong induction proof in Example 4. That this 
alternative form is equivalent to strong induction is left as Exercise 28. 


Prove that every amount of postage of 12 cents or more can be formed using just 4-cent and 
5-cent stamps. 


Solution: We will prove this result using the principle of mathematical induction. Then we will 
present a proof using strong induction. Let P(n) be the statement that postage of n cents can be 
formed using 4-cent and 5-cent stamps. 

We begin by using the principle of mathematical induction. 


BASIS STEP: Postage of 12 cents can be formed using three 4-cent stamps. 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(x) is true. That is, under 
this hypothesis, postage of k cents can be formed using 4-cent and 5-cent stamps. To complete 
the inductive step, we need to show that when we assume P(k) is true, then P(k + 1) is also 
true where k > 12. That is, we need to show that if we can form postage of k cents, then we can 
form postage of k + 1 cents. So, assume the inductive hypothesis is true; that is, assume that 
we can form postage of k cents using 4-cent and 5-cent stamps. We consider two cases, when at 
least one 4-cent stamp has been used and when no 4-cent stamps have been used. First, suppose 
that at least one 4-cent stamp was used to form postage of k cents. Then we can replace this 
stamp with a 5-cent stamp to form postage of k + 1 cents. But if no 4-cent stamps were used, 
we can form postage of k cents using only 5-cent stamps. Moreover, because k > 12, we needed 
at least three 5-cent stamps to form postage of k cents. So, we can replace three 5-cent stamps 
with four 4-cent stamps to form postage of k + 1 cents. This completes the inductive step. 


Because we have completed the basis step and the inductive step, we know that P(n) is 
true for alln > 12. That is, we can form postage of n cents, where n > 12 using just 4-cent and 
5-cent stamps. This completes the proof by mathematical induction. 


Next, we will use strong induction to prove the same result. In this proof, in the basis step 
we show that P(12), P(13), P(14), and P(15) are true, that is, that postage of 12, 13, 14, 
or 15 cents can be formed using just 4-cent and 5-cent stamps. In the inductive step we show 
how to get postage of k + 1 cents fork > 15 from postage of k — 3 cents. 


BASIS STEP: We can form postage of 12, 13, 14, and 15 cents using three 4-cent stamps, two 
4-cent stamps and one 5-cent stamp, one 4-cent stamp and two 5-cent stamps, and three 5-cent 
stamps, respectively. This shows that P(12), P(13), P(14), and P(15) are true. This completes 
the basis step. 
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INDUCTIVE STEP: The inductive hypothesis is the statement that P(j) is true for 12 < j <k, 
where k is an integer with k > 15. To complete the inductive step, we assume that we can form 
postage of j cents, where 12 < 7 < k. Weneed to show that under the assumption that P(k + 1) 
is true, we can also form postage of k + | cents. Using the inductive hypothesis, we can assume 
that P(k — 3) is true because k — 3 > 12, that is, we can form postage of k — 3 cents using 
just 4-cent and 5-cent stamps. To form postage of k + | cents, we need only add another 4-cent 
stamp to the stamps we used to form postage of k — 3 cents. That is, we have shown that if the 
inductive hypothesis is true, then P(k + 1) is also true. This completes the inductive step. 


Because we have completed the basis step and the inductive step of a strong induction proof, 
we know by strong induction that P (7) is true for all integers n with n > 12. That is, we know 
that every postage of n cents, where n is at least 12, can be formed using 4-cent and 5-cent 
stamps. This finishes the proof by strong induction. 

(There are other ways to approach this problem besides those described here. Can you find 
a solution that does not use mathematical induction?) 


Using Strong Induction in Computational Geometry 


Our next example of strong induction will come from computational geometry, the part of 
discrete mathematics that studies computational problems involving geometric objects. Compu- 
tational geometry is used extensively in computer graphics, computer games, robotics, scientific 
calculations, and a vast array of other areas. Before we can present this result, we introduce 
some terminology, possibly familiar from earlier studies in geometry. 

A polygon is a closed geometric figure consisting of a sequence of line segments 
S1,52,.-.-, Sn, called sides. Each pair of consecutive sides, sj; and sj41, i= 1,2,...,n—1, 
as well as the last side s, and the first side s;, of the polygon meet at a common endpoint, 
called a vertex. A polygon is called simple if no two nonconsecutive sides intersect. Every sim- 
ple polygon divides the plane into two regions: its interior, consisting of the points inside the 
curve, and its exterior, consisting of the points outside the curve. This last fact is surprisingly 
complicated to prove. It is a special case of the famous Jordan curve theorem, which tells us 
that every simple curve divides the plane into two regions; see [Or00], for example. 

A polygon is called convex if every line segment connecting two points in the interior of the 
polygon lies entirely inside the polygon. (A polygon that is not convex is said to be nonconvex.) 
Figure | displays some polygons; polygons (a) and (b) are convex, but polygons (c) and (d) are 
not. A diagonal of a simple polygon is a line segment connecting two nonconsecutive vertices of 
the polygon, and a diagonal is called an interior diagonal if it lies entirely inside the polygon, ex- 
cept for its endpoints. For example, in polygon (d), the line segment connecting a and f is an inte- 
rior diagonal, but the line segment connecting a and d is a diagonal that is not an interior diagonal. 

One of the most basic operations of computational geometry involves dividing a simple 
polygon into triangles by adding nonintersecting diagonals. This process is called triangulation. 
Note that a simple polygon can have many different triangulations, as shown in Figure 2. Perhaps 
the most basic fact in computational geometry is that it is possible to triangulate every simple 
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ad is not an interior diagonal 
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FIGURE 1 Convex and Nonconvex Polygons. 


THEOREM 1 


LEMMA 1 
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Two different triangulations of a simple polygon with seven sides into five triangles, 
shown with dotted lines and with dashed lines, respectively 


FIGURE 2. Triangulations of a Polygon. 


polygon, as we state in Theorem 1. Furthermore, this theorem tells us that every triangulation 
of a simple polygon with n sides includes n — 2 triangles. 


A simple polygon with n sides, where n is an integer with n > 3, can be triangulated into 
n — 2 triangles. 


It seems obvious that we should be able to triangulate a simple polygon by successively 
adding interior diagonals. Consequently, a proof by strong induction seems promising. However, 
such a proof requires this crucial lemma. 


Every simple polygon with at least four sides has an interior diagonal. 


Although Lemma | seems particularly simple, it is surprisingly tricky to prove. In fact, as 
recently as 30 years ago, a variety of incorrect proofs thought to be correct were commonly seen 
in books and articles. We defer the proof of Lemma | until after we prove Theorem 1. It is not 
uncommon to prove a theorem pending the later proof of an important lemma. 


Proof (of Theorem 1): We will prove this result using strong induction. Let T(n) be the 
statement that every simple polygon with n sides can be triangulated into n — 2 triangles. 


BASIS STEP: T (3) is true because a simple polygon with three sides is a triangle. We do not need 
to add any diagonals to triangulate a triangle; it is already triangulated into one triangle, itself. 
Consequently, every simple polygon withn = 3 has can be triangulated inton —2=3-—2= 1 
triangle. 


INDUCTIVE STEP: For the inductive hypothesis, we assume that T(j) is true for all 
integers j with 3 < j <k. That is, we assume that we can triangulate a simple polygon 
with j sides into 7 — 2 triangles whenever 3 < j < k. To complete the inductive step, we 
must show that when we assume the inductive hypothesis, P(k + 1) is true, that is, that every 
simple polygon with k + 1 sides can be triangulated into (k + 1) — 2 =k — | triangles. 

So, suppose that we have a simple polygon P withk + | sides. Becausek + 1 > 4, Lemma 1 
tells us that P has an interior diagonal ab. Now, ab splits P into two simple polygons Q, with 
s sides, and R, with t sides. The sides of Q and R are the sides of P, together with the side 
ab, which is a side of both Q and R. Note that 3 < 5 <k and 3 <t <k because both Q 
and R have at least one fewer side than P does (after all, each of these is formed from P 
by deleting at least two sides and replacing these sides by the diagonal ab). Furthermore, the 
number of sides of P is two less than the sum of the numbers of sides of Q and the number of 
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T is the triangle abc 
p is the vertex of P inside T such that the Z bap is smallest 


bp must be an interior diagonal of P 


Cc 


FIGURE 3 Constructing an Interior Diagonal of a Simple Polygon. 


sides of R, because each side of P is a side of either Q or of R, but not both, and the diagonal 
ab is a side of both Q and R, but not P. Thatis,k + 1=s+t—2. 

We now use the inductive hypothesis. Because both 3 < s < kand3 < t < k, by the induc- 
tive hypothesis we can triangulate Q and R into s — 2 and t — 2 triangles, respectively. Next, 
note that these triangulations together produce a triangulation of P. (Each diagonal added to 
triangulate one of these smaller polygons is also a diagonal of P.) Consequently, we can trian- 
gulate P into a total of (s — 2) + (t-— 2) =s+t-—4= (k+ 1) —2 triangles. This completes 
the proof by strong induction. That is, we have shown that every simple polygon with n sides, 
where n > 3, can be triangulated into n — 2 triangles. | 

We now return to our proof of Lemma 1. We present a proof published by Chung-Wu Ho 
[Ho75]. Note that although this proof may be omitted without loss of continuity, it does provide 
a correct proof of a result proved incorrectly by many mathematicians. 


Proof: Suppose that P is a simple polygon drawn in the plane. Furthermore, suppose that b is 
the point of P or in the interior of P with the least y-coordinate among the vertices with the 
smallest x-coordinate. Then b must be a vertex of P, for if it is an interior point, there would 
have to be a vertex of P with a smaller x-coordinate. Two other vertices each share an edge 
with b, say a and c. It follows that the angle in the interior of P formed by ab and bc must be 
less than 180 degrees (otherwise, there would be points of P with smaller x-coordinates than b). 

Now let T be the triangle Aabc. If there are no vertices of P on or inside T, we can 
connect a and c to obtain an interior diagonal. On the other hand, if there are vertices of P 
inside T, we will find a vertex p of P on or inside T such that bp is an interior diagonal. (This 
is the tricky part. Ho noted that in many published proofs of this lemma a vertex p was found 
such that bp was not necessarily an interior diagonal of P. See Exercise 21.) The key is to 
select a vertex p such that the angle Zbap is smallest. To see this, note that the ray starting 
at a and passing through p hits the line segment bc at a point, say q. It then follows that the 
triangle Abaq cannot contain any vertices of P in its interior. Hence, we can connect b and p 
to produce an interior diagonal of P. Locating this vertex p is illustrated in Figure 3. 5] 


Proofs Using the Well-Ordering Property 


The validity of both the principle of mathematical induction and strong induction follows from 
a fundamental axiom of the set of integers, the well-ordering property (see Appendix 1). The 
well-ordering property states that every nonempty set of nonnegative integers has a least element. 
We will show how the well-ordering property can be used directly in proofs. Furthermore, it 
can be shown (see Exercises 41, 42, and 43) that the well-ordering property, the principle of 
mathematical induction, and strong induction are all equivalent. That is, the validity of each of 
these three proof techniques implies the validity of the other two techniques. In Section 5.1 we 
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showed that the principle of mathematical induction follows from the well-ordering property. 
The other parts of this equivalence are left as Exercises 31, 42, and 43. 


THE WELL-ORDERING PROPERTY Every nonempty set of nonnegative integers has a 
least element. 


The well-ordering property can often be used directly in proofs. 


Use the well-ordering property to prove the division algorithm. Recall that the division algorithm 
states that if a is an integer and d is a positive integer, then there are unique integers g and r 


Solution: Let S be the set of nonnegative integers of the form a — dq, where q is an integer. This 
set is nonempty because —dq can be made as large as desired (taking q to be a negative integer 
with large absolute value). By the well-ordering property, S has a least element r = a — dqo. 
The integer r is nonnegative. It is also the case that r < d. If it were not, then there would 
be a smaller nonnegative element in S, namely, a — d(go + 1). To see this, suppose that r > d. 
Because a = dqy +1, it follows that a — d(go + 1) = (a —dqo) —d =r —d > 0. Conse- 
quently, there are integers g and r with 0 < r < d. The proof that g and r are unique is left as 
Exercise 37. < 


In a round-robin tournament every player plays every other player exactly once and each match 
, Pm forma cycle if p; beats p2, p2 
, Pm—1 beats pm, and pm beats p;. Use the well-ordering principle to show that if 
there is a cycle of length m (m > 3) among the players in a round-robin tournament, there must 


Solution: We assume that there is no cycle of three players. Because there is at least one cycle 
in the round-robin tournament, the set of all positive integers n for which there is a cycle of 
length n is nonempty. By the well-ordering property, this set of positive integers has a least 
element k, which by assumption must be greater than three. Consequently, there exists a cycle 


Because there is no cycle of three players, we know that k > 3. Consider the first three 
elements of this cycle, p;, p2, and p3. There are two possible outcomes of the match between 
pi and p3. If p3 beats py, it follows that pj, p2, p3 is a cycle of length three, contradicting 
our assumption that there is no cycle of three players. Consequently, it must be the case that 
, De to obtain the 
, Dk of length k — 1, contradicting the assumption that the smallest cycle 


EXAMPLE 5 

withO <r <danda=dq+r. 
Extra p> 
Examples 2 

EXAMPLE 6 
has a winner and a loser. We say that the players p;, p2,... 
beats p3,... 
be a cycle of three of these players. 
of players p1, p2, P3,---, pk and no shorter cycle exists. 
P, beats p3. This means that we can omit p2 from the cycle pj, p2, p3,..- 
cycle pj, p3, P4,--- 
has length k. We conclude that there must be a cycle of length three. 

Exercises 


1. Use strong induction to show that if you can run one mile 


parts of this exercise outline a strong induction proof that 


or two miles, and if you can always run two more miles 
once you have run a specified number of miles, then you 
can run any number of miles. 


. Use strong induction to show that all dominoes fall in an 
infinite arrangement of dominoes if you know that the 
first three dominoes fall, and that when a domino falls, 
the domino three farther down in the arrangement also 
falls. 


. Let P(n) be the statement that a postage of n cents can be 
formed using just 3-cent stamps and 5-cent stamps. The 


P(n) is true forn > 8. 

a) Show that the statements P(8), P(9), and P(10) are 
true, completing the basis step of the proof. 

b) What is the inductive hypothesis of the proof? 

c) What do you need to prove in the inductive step? 

d) Complete the inductive step for k > 10. 

e) Explain why these steps show that this statement is 
true whenever n > 8. 


. Let P(n) be the statement that a postage of n cents can be 


formed using just 4-cent stamps and 7-cent stamps. The 
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*9, 


10. 


11. 


parts of this exercise outline a strong induction proof that 
P(n) is true forn > 18. 


a) Show statements P(18), P(19), P(20), and P(21) 
are true, completing the basis step of the proof. 


b) What is the inductive hypothesis of the proof? 
c) What do you need to prove in the inductive step? 
d) Complete the inductive step for k > 21. 


e) Explain why these steps show that this statement is 
true whenever n > 18. 


. a) Determine which amounts of postage can be formed 


using just 4-cent and 11-cent stamps. 


b) Prove your answer to (a) using the principle of math- 
ematical induction. Be sure to state explicitly your 
inductive hypothesis in the inductive step. 


c) Prove your answer to (a) using strong induction. How 
does the inductive hypothesis in this proof differ from 
that in the inductive hypothesis for a proof using math- 
ematical induction? 


. a) Determine which amounts of postage can be formed 


using just 3-cent and 10-cent stamps. 


b) Prove your answer to (a) using the principle of math- 
ematical induction. Be sure to state explicitly your 
inductive hypothesis in the inductive step. 


c) Prove your answer to (a) using strong induction. How 
does the inductive hypothesis in this proof differ from 
that in the inductive hypothesis for a proof using math- 
ematical induction? 


. Which amounts of money can be formed using just two- 


dollar bills and five-dollar bills? Prove your answer using 
strong induction. 


. Suppose that a store offers gift certificates in denomina- 


tions of 25 dollars and 40 dollars. Determine the possible 
total amounts you can form using these gift certificates. 
Prove your answer using strong induction. 


Use strong induction to prove that V2 is irrational. [Hint: 
Let P(n) be the statement that /2 4 n/b for any positive 
integer b.] 


Assume that a chocolate bar consists of n squares ar- 
ranged in a rectangular pattern. The entire bar, a smaller 
rectangular piece of the bar, can be broken along a vertical 
or a horizontal line separating the squares. Assuming that 
only one piece can be broken at a time, determine how 
many breaks you must successively make to break the bar 
into n separate squares. Use strong induction to prove 
your answer. 


Consider this variation of the game of Nim. The game 
begins with n matches. Two players take turns removing 
matches, one, two, or three at a time. The player remov- 
ing the last match loses. Use strong induction to show 
that if each player plays the best strategy possible, the 
first player wins if n = 47, 47 + 2, or 47 +3 for some 
nonnegative integer j and the second player wins in the 
remaining case when n = 47 + 1 for some nonnegative 
integer j. 


12. 


* 13. 


14. 


15. 


*16. 


17. 


* 18. 


*19, 


Use strong induction to show that every positive integer n 
can be written as a sum of distinct powers of two, that is, 
as asum of a subset of the integers 29 — 1, 2i= Di 22 = 4, 
and so on. [Hint: For the inductive step, separately con- 
sider the case where k + 1 is even and where it is odd. 
When it is even, note that (k + 1)/2 is an integer.] 

A jigsaw puzzle is put together by successively joining 
pieces that fit together into blocks. A move is made each 
time a piece is added to a block, or when two blocks 
are joined. Use strong induction to prove that no matter 
how the moves are carried out, exactly n — 1 moves are 
required to assemble a puzzle with n pieces. 


Suppose you begin with a pile of n stones and split this 
pile into n piles of one stone each by successively split- 
ting a pile of stones into two smaller piles. Each time you 
split a pile you multiply the number of stones in each 
of the two smaller piles you form, so that if these piles 
have r and s stones in them, respectively, you compute 
rs. Show that no matter how you split the piles, the sum 
of the products computed at each step equals n(n — 1)/2. 


Prove that the first player has a winning strategy for the 
game of Chomp, introduced in Example 12 in Section 1.8, 
if the initial board is square. [Hint: Use strong induction 
to show that this strategy works. For the first move, the 
first player chomps all cookies except those in the left and 
top edges. On subsequent moves, after the second player 
has chomped cookies on either the top or left edge, the 
first player chomps cookies in the same relative positions 
in the left or top edge, respectively. ] 


Prove that the first player has a winning strategy for the 
game of Chomp, introduced in Example 12 in Section 1.8, 
if the initial board is two squares wide, that is, a 2 x n 
board. [Hint: Use strong induction. The first move of the 
first player should be to chomp the cookie in the bottom 
row at the far right.] 


Use strong induction to show that if a simple polygon 
with at least four sides is triangulated, then at least two 
of the triangles in the triangulation have two sides that 
border the exterior of the polygon. 


Use strong induction to show that when a simple poly- 
gon P with consecutive vertices vj, v2, ..., Vy, is trian- 
gulated into n — 2 triangles, the n — 2 triangles can be 
numbered 1, 2,..., — 2 so that v; is a vertex of triangle 
i fori =1,2,...,n—2. 

Pick’s theorem says that the area of a simple poly- 
gon P in the plane with vertices that are all lattice 
points (that is, points with integer coordinates) equals 
I(P)+B(P)/2—1, where /(P) and B(P) are the 
number of lattice points in the interior of P and on the 
boundary of P, respectively. Use strong induction on the 
number of vertices of P to prove Pick’s theorem. [Hint: 
For the basis step, first prove the theorem for rectangles, 
then for right triangles, and finally for all triangles by 
noting that the area of a triangle is the area of a larger 
rectangle containing it with the areas of at most three tri- 
angles subtracted. For the inductive step, take advantage 
of Lemma 1!.] 


** 20. 


21. 


Suppose that P is a simple polygon with vertices 
V1, V2,..., V, listed so that consecutive vertices are con- 
nected by an edge, and v, and v,, are connected by an edge. 
A vertex v; is called an ear if the line segment connecting 
the two vertices adjacent to v; is an interior diagonal of the 
simple polygon. Two ears v; and v; are called nonover- 
lapping if the interiors of the triangles with vertices v; 
and its two adjacent vertices and v; and its two adjacent 
vertices do not intersect. Prove that every simple polygon 
with at least four vertices has at least two nonoverlapping 
ears. 

In the proof of Lemma 1 we mentioned that many in- 
correct methods for finding a vertex p such that the 
line segment bp is an interior diagonal of P have been 
published. This exercise presents some of the incorrect 
ways p has been chosen in these proofs. Show, by con- 
sidering one of the polygons drawn here, that for each of 
these choices of p, the line segment bp is not necessarily 
an interior diagonal of P. 


a) p isthe vertex of P such that the angle Zabp is small- 
est. 

b) p is the vertex of P with the least x-coordinate (other 
than b). 

c) p is the vertex of P that is closest to b. 


c 


Exercises 22 and 23 present examples that show inductive 
loading can be used to prove results in computational geom- 


etry. 
#22. 


23. 


Let P(n) be the statement that when nonintersecting di- 
agonals are drawn inside a convex polygon with n sides, 
at least two vertices of the polygon are not endpoints of 
any of these diagonals. 


a) Show that when we attempt to prove P (n) for all inte- 
gersn withn > 3 using strong induction, the inductive 


step does not go through. 
b) Show that we can prove that P (7) is true for all inte- 


gers n with n > 3 by proving by strong induction the 
stronger assertion Q(n), forn > 4, where Q(n) states 
that whenever nonintersecting diagonals are drawn in- 
side a convex polygon with n sides, at least two non- 
adjacent vertices are not endpoints of any of these 
diagonals. 
Let E(n) be the statement that in a triangulation of a sim- 
ple polygon with n sides, at least one of the triangles in 
the triangulation has two sides bordering the exterior of 
the polygon. 


*24, 


25. 


26. 


27. 


28. 


29. 
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a) Explain where a proof using strong induction that 
E(n) is true for allintegersn > 4runs into difficulties. 


b) Show that we can prove that F(n) is true for all inte- 
gersn > 4by proving by strong induction the stronger 
statement T (1) for all integers n > 4, which states that 
in every triangulation of a simple polygon, at least two 
of the triangles in the triangulation have two sides bor- 
dering the exterior of the polygon. 


A stable assignment, defined in the preamble to Exer- 
cise 60 in Section 3.1, is called optimal for suitors if no 
stable assignment exists in which a suitor is paired with 
a suitee whom this suitor prefers to the person to whom 
this suitor is paired in this stable assignment. Use strong 
induction to show that the deferred acceptance algorithm 
produces a stable assignment that is optimal for suitors. 


Suppose that P (7) is a propositional function. Determine 

for which positive integers n the statement P(n) must be 

true, and justify your answer, if 

a) P(1) is true; for all positive integers n, if P (1) is true, 
then P(n + 2) is true. 

b) P(1) and P(2) are true; for all positive integers n, if 
P(n) and P(n + 1) are true, then P(n + 2) is true. 

c) P(1) is true; for all positive integers n, if P (7m) is true, 
then P(2n) is true. 

d) P(1) is true; for all positive integers n, if P (1) is true, 
then P(n + 1) is true. 

Suppose that P (7) is a propositional function. Determine 

for which nonnegative integers n the statement P(n) must 

be true if 

a) P(O) is true; for all nonnegative integers n, if P(n) is 
true, then P(n + 2) is true. 

b) P(O) is true; for all nonnegative integers n, if P(n) is 
true, then P(n + 3) is true. 

c) P(O) and P(1) are true; for all nonnegative integers n, 
if P(n) and P(n + 1) are true, then P(n + 2) is true. 

d) P(O) is true; for all nonnegative integers n, if P(n) is 
true, then P(n + 2) and P(n + 3) are true. 

Show that if the statement P (7) is true for infinitely many 

positive integers n and P(n + 1) > P(n) is true for all 

positive integers n, then P(7) is true for all positive inte- 

gers n. 


Let b be a fixed integer and j a fixed positive inte- 
ger. Show that if P(b), P(b+1),..., P(b+ j) are true 
and[P(b) A P(b+ 1) A---A P(k)] > P(k + 1) istrue 
for every integer k > b+ j, then P(n) is true for all 
integers n withn > b. 


What is wrong with this “proof” by strong induction? 
“Theorem” For every nonnegative integer n, Sn = 0. 
Basis Step: 5-0 = 0. 

Inductive Step: Suppose that 5j =0 for all nonneg- 
ative integers j with O< j <k. Writek+1=i+ 7, 
where i and 7 are natural numbers less than k + 1. By the 


inductive hypothesis, 5(k + 1) =5@+ j)=5i+5j= 
0+0=0. 
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* 30. 


*31. 


32. 


33. 


34. 


* 35. 


* 36. 


Find the flaw with the following “proof” that a” = 1 for 
all nonnegative integers n, whenever a is a nonzero real 
number. 


Basis Step: a° = 1 is true by the definition of a°. 
Inductive Step: Assume that a/ = 1 for all nonnegative 
integers j with j < k. Then note that 

k . gk 


k+1 a’:a 
ak 


1-1 
SS = 1 


Show that strong induction is a valid method of proof by 
showing that it follows from the well-ordering property. 


Find the flaw with the following “proof” that every 
postage of three cents or more can be formed using just 
three-cent and four-cent stamps. 


Basis Step: We can form postage of three cents with a 
single three-cent stamp and we can form postage of four 
cents using a single four-cent stamp. 


Inductive Step: Assume that we can form postage 
of j cents for all nonnegative integers j with j < k us- 
ing just three-cent and four-cent stamps. We can then 
form postage of k + 1 cents by replacing one three-cent 
stamp with a four-cent stamp or by replacing two four- 
cent stamps by three three-cent stamps. 


Show that we can prove that P(n, k) is true for all pairs 

of positive integers n and k if we show 

a) P(i,1) is true and P(n,k) >[P(n+I1,k)A 
P(n, k + 1)] is true for all positive integers n and k. 

b) P(,k) is true for all positive integers k, and 
P(n,k) > P(n+1,k) is true for all positive inte- 
gersn and k. 

c) P(n,1) is true for all positive integers n, and 
P(n,k) > P(n,k +1) is true for all positive inte- 
gers n and k. 

Prove that Do. /G+D(G+2)---G+k-D= 

n(n+ 1)(n+2)---(7+k)/(k + 1) for all positive inte- 

gers k and n. [Hint: Use a technique from Exercise 33.] 

Show thatifaj, a2,..., Gd, aren distinct real numbers, ex- 

actly n — | multiplications are used to compute the prod- 

uct of these m numbers no matter how parentheses are 
inserted into their product. [Hint: Use strong induction 
and consider the last multiplication. ] 

The well-ordering property can be used to show that there 

is a unique greatest common divisor of two positive in- 

tegers. Let a and b be positive integers, and let S be 


37. 


38. 


** 39, 


40. 


*41, 


* 42, 


* 43. 


the set of positive integers of the form as + bt, where s 
and ¢ are integers. 


a) Show that S is nonempty. 

b) Use the well-ordering property to show that S has a 
smallest element c. 

c) Show that if d is acommon divisor of a and b, then d 
is a divisor of c. 

d) Show that c | a and c | b. [Hint: First, assume that 
c J a. Thena = qc +r, where 0 < r < c. Show that 
r € S, contradicting the choice of c.] 

e) Conclude from (c) and (d) that the greatest common 
divisor of a and b exists. Finish the proof by showing 
that this greatest common divisor is unique. 


Let a be an integer and d be a positive integer. Show 
that the integers g andr witha = dq +rand0 <r <d, 
which were shown to exist in Example 5, are unique. 


Use mathematical induction to show that a rectangu- 
lar checkerboard with an even number of cells and two 
squares missing, one white and one black, can be covered 
by dominoes. 


Can you use the well-ordering property to prove the state- 
ment: “Every positive integer can be described using no 
more than fifteen English words”? Assume the words 
come from a particular dictionary of English. [Hint: Sup- 
pose that there are positive integers that cannot be de- 
scribed using no more than fifteen English words. By 
well ordering, the smallest positive integer that cannot 
be described using no more than fifteen English words 
would then exist.] 


Use the well-ordering principle to show that if x and y 
are real numbers with x < y, then there is a rational 
number r with x <r < y. [Hint: Use the Archimedean 
property, given in Appendix 1, to find a positive 
integer A with A > 1/(y — x). Then show that there is 
a rational number r with denominator A between x and 
y by looking at the numbers |x| + j/A, where j is a 
positive integer. ] 

Show that the well-ordering property can be proved when 
the principle of mathematical induction is taken as an ax- 
iom. 


Show that the principle of mathematical induction and 
strong induction are equivalent; that is, each can be shown 
to be valid from the other. 


Show that we can prove the well-ordering property when 
we take strong induction as an axiom instead of taking 
the well-ordering property as an axiom. 


Recursive Definitions and Structural Induction 


Introduction 


Sometimes it is difficult to define an object explicitly. However, it may be easy to define this 
object in terms of itself. This process is called recursion. For instance, the picture shown in 
Figure | is produced recursively. First, an original picture is given. Then a process of successively 
superimposing centered smaller pictures on top of the previous pictures is carried out. 
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FIGURE 1 A Recursively Defined Picture. 


We can use recursion to define sequences, functions, and sets. In Section 2.4, and in most 
beginning mathematics courses, the terms of a sequence are specified using an explicit formula. 
For instance, the sequence of powers of 2 is given by a, = 2” forn = 0, 1, 2,.... Recall from 
Section 2.4 that we can also define a sequence recursively by specifying how terms of the 
sequence are found from previous terms. The sequence of powers of 2 can also be defined 
by giving the first term of the sequence, namely, ag = 1, and a rule for finding a term of the 
sequence from the previous one, namely, a,4; = 2a, forn =0,1,2,.... When we define a 
sequence recursively by specifying how terms of the sequence are found from previous terms, 
we can use induction to prove results about the sequence. 

When we define a set recursively, we specify some initial elements in a basis step and 
provide a rule for constructing new elements from those we already have in the recur- 
sive step. To prove results about recursively defined sets we use a method called structural 
induction. 


Recursively Defined Functions 


We use two steps to define a function with the set of nonnegative integers as its domain: 
BASIS STEP: Specify the value of the function at zero. 


RECURSIVE STEP: Give a rule for finding its value at an integer from its values at smaller 
integers. 


Such a definition is called a recursive or inductive definition. Note that a function f(n) from 
the set of nonnegative integers to the set of a real numbers is the same as a sequence dg, dj,... 
where a; is a real number for every nonnegative integer i. So, defining a real-valued sequence 
ao, 41, ... using a recurrence relation, as was done in Section 2.4, is the same as defining a 
function from the set of nonnegative integers to the set of real numbers. 
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EXAMPLE 1 
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EXAMPLE 2 


EXAMPLE 3 


Suppose that f is defined recursively by 


f (0) = 3, 
f(n+1) =2f(n) +3. 


Find f(1), f(2), f(3), and f (4). 
Solution: From the recursive definition it follows that 


fQ) =2f0)+3=2-34+3=9, 

f@=2f0) +3 =2-943=21, 

fGB) =2f2)4+3=2-2143=45, 

f4@ =2f8)4+3=2-454+3=93. P 


Recursively defined functions are well defined. That is, for every positive integer, the 
value of the function at this integer is determined in an unambiguous way. This means 
that given any positive integer, we can use the two parts of the definition to find the value 
of the function at that integer, and that we obtain the same value no matter how we ap- 
ply the two parts of the definition. This is a consequence of the principle of mathemati- 
cal induction. (See Exercise 56.) Additional examples of recursive definitions are given in 
Examples 2 and 3. 


Give a recursive definition of a”, where a is a nonzero real number and n is a nonnegative 
integer. 


Solution: The recursive definition contains two parts. First a° is specified, namely, a® = 1. Then 
the rule for finding a"t! from a", namely, a"+! —q.aq",forn = 0, 1,2,3,..., is given. These 


two equations uniquely define a” for all nonnegative integers n. < 


Give a recursive definition of 


Solution: The first part of the recursive definition is 


0 
yaa, 
k=0 


The second part is 


n+1 


n 
> a= (>: a) + an+1- 
k=0 k=0 


< 


In some recursive definitions of functions, the values of the function at the first k positive 
integers are specified, and a rule is given for determining the value of the function at larger inte- 
gers from its values at some or all of the preceding k integers. That recursive definitions defined 
in this way produce well-defined functions follows from strong induction (see Exercise 57). 
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Recall from Section 2.4 that the Fibonacci numbers, fo, fi, f2,..., are defined by the 
equations fp = 0, fi = 1, and 


Sn = fn—-1 + fn-2 
for n = 2,3,4,.... [We can think of the Fibonacci number f,, either as the nth term of the 
sequence of Fibonacci numbers fo, /1, ... or as the value at the integer n of a function f(n).] 


We can use the recursive definition of the Fibonacci numbers to prove many properties of 
these numbers. We give one such property in Example 4. 


Show that whenever n > 3, f, > a”? where a = (1 + 5) 2: 


olution: We can use strong induction to prove this inequality. Let P(n) be the statemen 
Solution: We trong induction to p th quality. Let P(n) be th t t 
fn > w"~?, We want to show that P(n) is true whenever n is an integer greater than or equal to 3. 


BASIS STEP: First, note that 
a<2=f3, ow =(34+V5)/2<3= fa, 


so P(3) and P(4) are true. 


INDUCTIVE STEP: Assume that P(j) is true, namely, that fj > a/~?, for all integers j with 
3 < j <k,wherek > 4. We must show that P(k + 1) is true, that is, that fgi1 > a‘k—! Because 
a is a solution of x2 — x — 1 = 0 (as the quadratic formula shows), it follows that a=atl. 
Therefore, 


ec ec 


By the inductive hypothesis, because k > 4, we have 


k-3 k-2 


fe-1 > Sk > @ 


Therefore, it follows that 


ae = a 
Siti = fie + fe-1 > a“ +a" =a” 


Hence, P(k + 1) is true. This completes the proof. < 


Remark: The inductive step shows that whenever k > 4, P(k + 1) follows from the assumption 
that P(j) is true for 3 < j <k. Hence, the inductive step does not show that P(3) > P(4). 
Therefore, we had to show that P (4) is true separately. 


We can now show that the Euclidean algorithm, introduced in Section 4.3, uses O(log b) 
divisions to find the greatest common divisor of the positive integers a and b, where a > b. 


LAME’S THEOREM Let a and b be positive integers with a > b. Then the number of 
divisions used by the Euclidean algorithm to find gcd(a, b) is less than or equal to five times 
the number of decimal digits in b. 
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Proof: Recall that when the Euclidean algorithm is applied to find gcd(a, b) with a > b, this 
sequence of equations (where a = ro and b = 71) is obtained. 


ro ="191 +12 O<r<ri, 
ry =12q2 + 73 O0<1r3</r2, 
Tn—2 =Tn-19n-1 + Tn O<rp <TVn-1; 


Tn—-1 =n n- 


Here n divisions have been used to find r, = gcd(a, b). Note that the quotients q1, 92, .--, Gn—1 
are all at least 1. Moreover, g, > 2, because r, < r,—1. This implies that 


l= fo, 
Mn—-\ = 2% = 2 fr = BB, 
PAs bh =I. 


ln 


V IV 


IV 


Tn-2 


r2 r3+7r4> fo-1+ fr—-2 = fn; 
b=r 272 FT = fn t+ fn-1 = fn4i- 


IV 


It follows that if n divisions are used by the Euclidean algorithm to find gcd(a, b) with a > b, 
then b > f,41. By Example 4 we know that f,41 > a”—! forn > 2, where a = (1+ /5) /2. 
Therefore, it follows that b > w”—!. Furthermore, because logjg @ © 0.208 > 1/5, we see that 


logigb > (n — 1) logyga > (n — 1)/5. 


Hence, n — 1 < 5- log;g 6. Now suppose that b has k decimal digits. Then b < 10* and 
logiy b < k. It follows that n — 1 < 5k, and because k is an integer, it follows that n < 5k. 
This finishes the proof. | 


Because the number of decimal digits in b, which equals |log;g b| + 1, is less than or equal 
to logj) b+ 1, Theorem 1 tells us that the number of divisions required to find gcd(a, b) with 


FIBONACCI (1170-1250) Fibonacci (short for filius Bonacci, or “son of Bonacci”) was also known as 
Leonardo of Pisa. He was born in the Italian commercial center of Pisa. Fibonacci was a merchant who traveled 
extensively throughout the Mideast, where he came into contact with Arabian mathematics. In his book Liber 
Abaci, Fibonacci introduced the European world to Arabic notation for numerals and algorithms for arithmetic. 
It was in this book that his famous rabbit problem (described in Section 8.1) appeared. Fibonacci also wrote 
books on geometry and trigonometry and on Diophantine equations, which involve finding integer solutions to 
equations. 
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a > bis less than or equal to 5(logj) b + 1). Because S5(log;, b + 1) is O(log b), we see that 
O (log b) divisions are used by the Euclidean algorithm to find gcd(a, b) whenever a > b. 


Recursively Defined Sets and Structures 


We have explored how functions can be defined recursively. We now turn our attention to how sets 
can be defined recursively. Just as in the recursive definition of functions, recursive definitions 
of sets have two parts, a basis step and a recursive step. In the basis step, an initial collection 
of elements is specified. In the recursive step, rules for forming new elements in the set from 
those already known to be in the set are provided. Recursive definitions may also include an 
exclusion rule, which specifies that a recursively defined set contains nothing other than those 
elements specified in the basis step or generated by applications of the recursive step. In our 
discussions, we will always tacitly assume that the exclusion rule holds and no element belongs 
to a recursively defined set unless it is in the initial collection specified in the basis step or can 
be generated using the recursive step one or more times. Later we will see how we can use a 
technique known as structural induction to prove results about recursively defined sets. 

Examples 5, 6, 8, and 9 illustrate the recursive definition of sets. In each example, we show 
those elements generated by the first few applications of the recursive step. 


Consider the subset S of the set of integers recursively defined by 
BASIS STEP: 3.€ S. 
RECURSIVE STEP: Ifx € Sandy € S,thenx+yeS. 


The new elements found to be in S are 3 by the basis step, 3 + 3 = 6at the first application of 
the recursive step, 3 + 6 = 6+3 = 9and6-+ 6 = 12 at the second application of the recursive 
step, and so on. We will show in Example 10 that S is the set of all positive multiples of 3. <4 


Recursive definitions play an important role in the study of strings. (See Chapter 13 for 
an introduction to the theory of formal languages, for example.) Recall from Section 2.4 that a 
string over an alphabet © is a finite sequence of symbols from ©. We can define &*, the set of 
strings over J, recursively, as Definition 1 shows. 


The set &* of strings over the alphabet & is defined recursively by 
BASIS STEP: 2» € &* (where A is the empty string containing no symbols). 
RECURSIVE STEP: If w € &* and x € &, then wx € X*. 


GABRIEL LAME (1795-1870) Gabriel Lamé entered the Ecole Polytechnique in 1813, graduating in 1817. 
He continued his education at the Ecole des Mines, graduating in 1820. 

In 1820 Lamé went to Russia, where he was appointed director of the Schools of Highways and Trans- 
portation in St. Petersburg. Not only did he teach, but he also planned roads and bridges while in Russia. He 
returned to Paris in 1832, where he helped found an engineering firm. However, he soon left the firm, accepting 
the chair of physics at the Ecole Polytechnique, which he held until 1844. While holding this position, he was 
active outside academia as an engineering consultant, serving as chief engineer of mines and participating in 
the building of railways. 

Lamé contributed original work to number theory, applied mathematics, and thermodynamics. His best- 


known work involves the introduction of curvilinear coordinates. His work on number theory includes proving Fermat’s last theorem 
for n = 7, as well as providing the upper bound for the number of divisions used by the Euclidean algorithm given in this text. 

In the opinion of Gauss, one of the most important mathematicians of all time, Lamé was the foremost French mathematician of 
his time. However, French mathematicians considered him too practical, whereas French scientists considered him too theoretical. 
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The basis step of the recursive definition of strings says that the empty string belongs to &*. 
The recursive step states that new strings are produced by adding a symbol from © to the end of 
strings in )*. At each application of the recursive step, strings containing one additional symbol 
are generated. 


If & = {0, 1}, the strings found to be in &*, the set of all bit strings, are 4, specified to be in X* 
in the basis step, 0 and 1 formed during the first application of the recursive step, 00, 01, 10, 
and 11 formed during the second application of the recursive step, and so on. < 


Recursive definitions can be used to define operations or functions on the elements of 
recursively defined sets. This is illustrated in Definition 2 of the concatenation of two strings 
and Example 7 concerning the length of a string. 


Two strings can be combined via the operation of concatenation. Let X be a set of symbols 
and &* the set of strings formed from symbols in ©. We can define the concatenation of two 
strings, denoted by -, recursively as follows. 


BASIS STEP: Ifw € &*, then w- A = w, where A is the empty string. 
RECURSIVE STEP: If wy € &* and wz € &* and x € B, then wy - (wox) = (wy - wWo)x. 


The concatenation of the strings w; and w2 is often written as w;w2 rather than w; - w2. By 
repeated application of the recursive definition, it follows that the concatenation of two strings 
w, and w2 consists of the symbols in w; followed by the symbols in w2. For instance, the 
concatenation of wy = abra and w2 = cadabra is w\w2 = abracadabra. 


Length of a String Give a recursive definition of /(w), the length of the string w. 


Solution: The length of a string can be recursively defined by 


1(A) = 0; 


(wx) =1(w) + lifw e€ X* andx € D. 4 


Another important use of recursive definitions is to define well-formed formulae of various 
types. This is illustrated in Examples 8 and 9. 


Well-Formed Formulae in Propositional Logic We can define the set of well-formed for- 
mulae in propositional logic involving T, F, propositional variables, and operators from the set 
{4,A,V,>, <<}. 


BASIS STEP: T, F, and s, where s is a propositional variable, are well-formed formulae. 


RECURSIVE STEP: If E and F are well-formed formulae, then (-E), (E A F), (EV F), 
(E — F), and (E < F) are well-formed formulae. 


For example, by the basis step we know that T, F, p, and g are well-formed formulae, 
where p and g are propositional variables. From an initial application of the recursive step, 
we know that (p V q), (p > F), (F > gq), and (¢ AF) are well-formed formulae. A sec- 
ond application of the recursive step shows that ((p Vg) > (q AF)), (¢ V (pV q)), and 
((p > F) — T) are well-formed formulae. We leave it to the reader to show that p— A q, 
pq”, and = A pq are not well-formed formulae, by showing that none can be obtained using 
the basis step and one or more applications of the recursive step. 
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EXAMPLE 9 Well-Formed Formulae of Operators and Operands We can define the set of well-formed 
formulae consisting of variables, numerals, and operators from the set {+, —, *, /, +} (where « 
denotes multiplication and + denotes exponentiation) recursively. 


BASIS STEP: x is a well-formed formula if x is a numeral or a variable. 


RECURSIVE STEP: If F and G are well-formed formulae, then (fF + G), (F — G), (F « G), 
(F/G), and (F + G) are well-formed formulae. 


For example, by the basis step we see that x, y, 0, and 3 are well-formed formulae (as is 
any variable or numeral). Well-formed formulae generated by applying the recursive step once 
include (x + 3), 3+y), @—y), 3-90), @ * 3), B* y), (3/0), @/y), Gtx), and (OF 3). 
Applying the recursive step twice shows that formulae such as ((x + 3) + 3) and (x — (3 * y)) 
are well-formed formulae. [Note that (3/0) is a well-formed formula because we are concerned 
only with syntax matters here.] We leave it to the reader to show that each of the formulae x3 +, 
y * +x, and « x/y is not a well-formed formula by showing that none of them can be obtained 
from the basis step and one or more applications of the recursive step. 


We will study trees extensively in Chapter 11. A tree is a special type of a graph; a graph 
is made up of vertices and edges connecting some pairs of vertices. We will study graphs in 
Chapter 10. We will briefly introduce them here to illustrate how they can be defined recursively. 


DEFINITION 3 The set of rooted trees, where a rooted tree consists of a set of vertices containing a distin- 
guished vertex called the root, and edges connecting these vertices, can be defined recursively 
by these steps: 


BASIS STEP: A single vertex r is a rooted tree. 
RECURSIVE STEP: Suppose that 7), T,..., 7, are disjoint rooted trees with roots 


r1,12,---;1n, Tespectively. Then the graph formed by starting with a root r, which is not 
in any of the rooted trees T;, 72, ..., T,, and adding an edge from r to each of the vertices 
r1,12,..-,1y, 1S also a rooted tree. 


In Figure 2 we illustrate some of the rooted trees formed starting with the basis step and applying 
the recursive step one time and two times. Note that infinitely many rooted trees are formed at 
each application of the recursive definition. 


Basis step e 
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FIGURE 2 _ Building Up Rooted Trees. 


352 5/ Induction and Recursion 


Basis step © 


Step | ° 


~ Lo * 


© LS OOMN LN AIL) 
(6? \KNANKYAN 
APALALAC 


FIGURE 3 Building Up Extended Binary Trees. 


DEFINITION 4 


Binary trees are a special type of rooted trees. We will provide recursive definitions of 
two types of binary trees—full binary trees and extended binary trees. In the recursive step of 
the definition of each type of binary tree, two binary trees are combined to form a new tree 
with one of these trees designated the left subtree and the other the right subtree. In extended 
binary trees, the left subtree or the right subtree can be empty, but in full binary trees this is not 
possible. Binary trees are one of the most important types of structures in computer science. In 
Chapter 11 we will see how they can be used in searching and sorting algorithms, in algorithms 
for compressing data, and in many other applications. We first define extended binary trees. 


The set of extended binary trees can be defined recursively by these steps: 
BASIS STEP: The empty set is an extended binary tree. 


RECURSIVE STEP: If T, and T> are disjoint extended binary trees, there is an extended 
binary tree, denoted by 7; - 72, consisting of a root r together with edges connecting the 
root to each of the roots of the left subtree 7; and the right subtree T> when these trees are 
nonempty. 


Figure 3 shows how extended binary trees are built up by applying the recursive step from one 
to three times. 

We now show how to define the set of full binary trees. Note that the difference between 
this recursive definition and that of extended binary trees lies entirely in the basis step. 


Basis step e 
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FIGURE 4 _ Building Up Full Binary Trees. 
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The set of full binary trees can be defined recursively by these steps: 
BASIS STEP: There is a full binary tree consisting only of a single vertex r. 


RECURSIVE STEP: If T; and 7) are disjoint full binary trees, there is a full binary tree, 
denoted by 7; - 72, consisting of a root r together with edges connecting the root to each of 
the roots of the left subtree 7; and the right subtree 7). 


Figure 4 shows how full binary trees are built up by applying the recursive step one and two 
times. 


Structural Induction 


To prove results about recursively defined sets, we generally use some form of mathematical 
induction. Example 10 illustrates the connection between recursively defined sets and mathe- 
matical induction. 


Show that the set S defined in Example 5 by specifying that 3 € S and thatifx € Sandy eS, 
then x + y € S, is the set of all positive integers that are multiples of 3. 


Solution: Let A be the set of all positive integers divisible by 3. To prove that A = S, we must 
show that A is a subset of S and that S is a subset of A. To prove that A is a subset of S, we 
must show that every positive integer divisible by 3 is in S. We will use mathematical induction 
to prove this. 

Let P(n) be the statement that 3n belongs to S. The basis step holds because by the first 
part of the recursive definition of S$, 3- 1 = 3 is in S. To establish the inductive step, assume 
that P(k) is true, namely, that 3k is in §. Because 3k is in S and because 3 is in S, it follows 
from the second part of the recursive definition of S that 3k + 3 = 3(k + 1) isalsoin S. 

To prove that S is a subset of A, we use the recursive definition of S. First, the basis step 
of the definition specifies that 3 is in S. Because 3 = 3 - 1, all elements specified to be in S in 
this step are divisible by 3 and are therefore in A. To finish the proof, we must show that all 
integers in S generated using the second part of the recursive definition are in A. This consists 
of showing that x + y isin A whenever x and y are elements of S also assumed to be in A. Now 
if x and y are both in A, it follows that 3 | x and 3 | y. By part (i) of Theorem 1 of Section 4.1, 
it follows that 3 | x + y, completing the proof. < 
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EXAMPLE 11 


In Example 10 we used mathematical induction over the set of positive integers and a 
recursive definition to prove a result about a recursively defined set. However, instead of using 
mathematical induction directly to prove results about recursively defined sets, we can use a more 
convenient form of induction known as structural induction. A proof by structural induction 
consists of two parts. These parts are 


BASIS STEP: Show that the result holds for all elements specified in the basis step of the 
recursive definition to be in the set. 


RECURSIVE STEP: Show that if the statement is true for each of the elements used to construct 
new elements in the recursive step of the definition, the result holds for these new elements. 


The validity of structural induction follows from the principle of mathematical induction 
for the nonnegative integers. To see this, let P(m) state that the claim is true for all elements 
of the set that are generated by n or fewer applications of the rules in the recursive step of 
a recursive definition. We will have established that the principle of mathematical induction 
implies the principle of structural induction if we can show that P(m) is true whenever n is a 
positive integer. In the basis step of a proof by structural induction we show that P (0) is true. 
That is, we show that the result is true of all elements specified to be in the set in the basis 
step of the definition. A consequence of the recursive step is that if we assume P(k) is true, 
it follows that P(k + 1) is true. When we have completed a proof using structural induction, 
we have shown that P (0) is true and that P(k) implies P(k + 1). By mathematical induction it 
follows that P(n) is true for all nonnegative integers n. This also shows that the result is true 
for all elements generated by the recursive definition, and shows that structural induction is a 
valid proof technique. 


EXAMPLES OF PROOFS USING STRUCTURAL INDUCTION _ Structural induction can 
be used to prove that all members of a set constructed recursively have a particular property. 
We will illustrate this idea by using structural induction to prove results about well-formed 
formulae, strings, and binary trees. For each proof, we have to carry out the appropriate basis 
step and the appropriate recursive step. For example, to use structural induction to prove a 
result about the set of well-formed formulae defined in Example 8, where we specify that T, F, 
and every propositional variable s are well-formed formulae and where we specify that if E 
and F are well-formed formulae, then (-E), (E A F), (EV F), (E — F), and (E <= F) are 
well-formed formulae, we need to complete this basis step and this recursive step. 


BASIS STEP: Show that the result is true for T, F, and s whenever s is a propositional variable. 
RECURSIVE STEP: Show that if the result is true for the compound propositions p and q, it 
is also true for (~p), (p V q), (p Aq), (p > q), and (p = q). 


Example | | illustrates how we can prove results about well-formed formulae using structural 
induction. 


Show that every well-formed formula for compound propositions, as defined in Example 8, 
contains an equal number of left and right parentheses. 


Solution: 


BASIS STEP: Each of the formula T, F, and s contains no parentheses, so clearly they contain 
an equal number of left and right parentheses. 


RECURSIVE STEP: Assume p and q are well-formed formulae each containing an equal 
number of left and right parentheses. That is, if /, and /, are the number of left parentheses 
in p and q, respectively, and ry, and rg are the number of right parentheses in p and q, respec- 
tively, then /, = rp, and /, = r,. To complete the inductive step, we need to show that each of 


EXAMPLE 12 


DEFINITION 6 
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(=p), (pV gq), (P Aq), (p > q), and (p = q) also contains an equal number of left and right 
parentheses. The number of left parentheses in the first of these compound propositions equals 
1, + 1 and in each of the other compound propositions equals /,, + J, + 1. Similarly, the number 
of right parentheses in the first of these compound propositions equals rp, + 1 and in each of the 
other compound propositions equals rp, + rg + 1. Because 1) = rp and Jy = rq, it follows that 
each of these compound expressions contains the same number of left and right parentheses. 
This completes the proof by structural induction. < 


Suppose that P (w) is a propositional function over the set of strings w € X*. To use structural 
induction to prove that P(w) holds for all strings w € 4*, we need to complete both a basis step 
and a recursive step. These steps are: 


BASIS STEP: Show that P(A) is true. 


RECURSIVE STEP: Assume that P (w) is true, where w € &*. Show that if x € X, then P(wx) 
must also be true. 


Example 12 illustrates how structural induction can be used in proofs about strings. 


Use structural induction to prove that /(xy) = /(x) +/(y), where x and y belong to &*, the set 
of strings over the alphabet ©. 


Solution: We will base our proof on the recursive definition of the set &* given in Definition 1 
and the definition of the length of a string in Example 7, which specifies that /(1) = 0 and 
l(wx) = l(w) + | whenw € &* andx € &. Let P(y) be the statement that /(xy) = I(x) + 1(y) 
whenever x belongs to X*. 


BASIS STEP: To complete the basis step, we must show that P(A) is true. That is, we must 
show that /(x.) = I(x) + 1(A) for all x € D*. Because /(xA) = I(x) = 1(x) +0 =1(x) +1(A) 
for every string x, it follows that P(A) is true. 

RECURSIVE STEP: To complete the inductive step, we assume that P(y) is true and 
show that this implies that P(ya) is true whenever a € &. What we need to show is that 
l(xya) =1(x) +l(ya) forevery a € &.To show this, note that by the recursive definition of /(w) 
(given in Example 7), we have /(xya) = /(xy) + 1 and/(ya) = /(y) + 1. And, by the inductive 
hypothesis, /(xy) = I(x) + 1(y). We conclude that I(xya) = I(x) + 1(y) + 1 = 1(x) + l(ya)4 


We can prove results about trees or special classes of trees using structural induction. For 
example, to prove a result about full binary trees using structural induction we need to complete 
this basis step and this recursive step. 


BASIS STEP: Show that the result is true for the tree consisting of a single vertex. 


RECURSIVE STEP: Show that if the result is true for the trees T,; and 7, then it is true for 
tree T; - T> consisting of a root r, which has 7} as its left subtree and 7) as its right subtree. 


Before we provide an example showing how structural induction can be used to prove a 
result about full binary trees, we need some definitions. We will recursively define the height 
h(T) and the number of vertices n(T) of a full binary tree T. We begin by defining the height 
of a full binary tree. 


We define the height h(T) of a full binary tree T recursively. 
BASIS STEP: The height of the full binary tree T consisting of only a root r is h(T) = 0. 


RECURSIVE STEP: If 7; and T) are full binary trees, then the full binary tree T = Tj - T> 
has height h(T) = 1 + max(A(T)), h(7>)). 
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THEOREM 2 


If we let n(T) denote the number of vertices in a full binary tree, we observe that n(T) satisfies 
the following recursive formula: 


BASIS STEP: The number of vertices n(T) of the full binary tree T consisting of only a root 
risn(T)=1. 


RECURSIVE STEP: If T; and T> are full binary trees, then the number of vertices of the full 
binary tree T = T - Ty isn(T) = 1+n(T,) +n(Tp). 


We now show how structural induction can be used to prove a result about full binary trees. 


If T is a full binary tree T, then n(T) < MED ee ile 


Proof: We prove this inequality using structural induction. 


BASIS STEP: For the full binary tree consisting of just the root r the result is true because 
n(T) = land A(T) = 0, so that n(T) = 1 < 2°! —-1=1. 


RECURSIVE STEP: For the inductive hypothesis we assume that n(7,) < 2’(7+! — 1 and 
n(Tn) < Qh(T2)+1 _ | whenever T, and T> are full binary trees. By the recursive formulae for 
n(T) and h(T) we have n(T) = 1+ n(7)) +n(1y) and h(T) = 1+ max(h(T)), h(T)). 

We find that 


n(T) =1+n(7T;) + n(th) by the recursive formula for n(T) 
< 14 (2h(T)+1 — 1) 4 (2hC2)+1 _ 1) _ by the inductive hypothesis 
29 max! OMe) because the sum of two terms is at most 2 

times the larger 
= 2. 2max(h(Ti),h(T2))+1 _ 4 because max(2*, 2”) = 2max(x.y) 
=2.2h(T) _ 4 by the recursive definition of h(T) 
= h(T)+1 _ 4. 
This completes the recursive step. | 


Generalized Induction 


We can extend mathematical induction to prove results about other sets that have the well- 
ordering property besides the set of integers. Although we will discuss this concept in detail in 
Section 9.6, we provide an example here to illustrate the usefulness of such an approach. 

As an example, note that we can define an ordering on N x N, the ordered pairs of non- 
negative integers, by specifying that (x1, y;) is less than or equal to (x2, y2) if either x; < x2, 
or xj = x2 and y; < yg; this is called the lexicographic ordering. The set N x N with this 
ordering has the property that every subset of N x N has a least element (see Exercise 53 
in Section 9.6). This implies that we can recursively define the terms a,,, with m € N and 
n €N, and prove results about them using a variant of mathematical induction, as illustrated in 
Example 13. 
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EXAMPLE 13 — Suppose that a,» is defined recursively for (m,n) € N x N by ao.o = O and 


AQn-in+1 ifn=Oandm > 0 
ann = 


Aamn-l +n ifn >0. 


Show that a@j)., = m-+n(n + 1)/2 for all (m,n) € N XN, that is, for all pairs of nonnegative 
integers. 


Solution: Wecan prove that a, = m + n(n + 1)/2 using a generalized version of mathematical 
induction. The basis step requires that we show that this formula is valid when (m, n) = (0, 0). 
The induction step requires that we show that if the formula holds for all pairs smaller than 
(m,n) in the lexicographic ordering of N x N, then it also holds for (m, n). 


BASIS STEP: Let (m,n) = (0, 0). Then by the basis case of the recursive definition of a,,_; 
we have ao,o = 0. Furthermore, when m =n = 0,m-+n(n+ 1)/2=0+ (0- 1)/2 = 0. This 
completes the basis step. 


INDUCTIVE STEP: Suppose that a7, =m’ +n'(n’+1)/2 whenever (m’,n’) is less 
than (m,n) in the lexicographic ordering of N x N. By the recursive definition, if n = 0, 
then @m.n = Am—1,n + 1. Because (m — 1,n) is smaller than (m,n), the inductive hypoth- 
esis tells us that @m—1.. =m —1+n(n+ 1)/2, so that dy, =m—1+nain+1)/24+1= 
m+n(n + 1)/2, giving us the desired equality. Now suppose thatn > 0,80 dm = 4mn—1 +N. 
Because (m,n — 1) is smaller than (m,n), the inductive hypothesis tells us that dy.,—1 = 
m+ (n — 1)n/2,s0dy.n = m+ (n— 1)n/2+n=m+ (n2—n+ 2n)/2=m+n(n+ 1)/2. 
This finishes the inductive step. < 


As mentioned, we will justify this proof technique in Section 9.6. 


Exercises 


1. Find f(1), f(2), f(), and f(4) if f (7) is defined recur- 5. Determine whether each of these proposed definitions is 


sively by f(0) = 1 and forn = 0,1,2,... 

a) fa+1)= fm) +2. 

b) fa+1)=3f(n). 

ec fath=2/™, 

d) f2t+)l = fa?+ fiat. 

. Find f(1), f(2), f(3), f(4), and f (5) if f(a) is defined 
recursively by f (0) = 3 and form = 0,1, 2,... 

a) fm+1)=-2f(n). 

b) fmt+1)=3f@) +7. 

o) fat l= fn? —2f@ -2. 

d) f+) = 34M, 

. Find f(2), f(3), f(4), and f(5) if f is defined recur- 
sively by f(0) = —1, f(1) = 2, and forn = 1,2,... 

a) fa+)D=fm4+3fa-1). 

b) fin+1) = fay fa — I). 

ce) f@tl=3f()’-—4fa— DD’. 

d) fat+)= fa—1)/f@). 

. Find f(2), f(3), f(4), and f(5) if f is defined recur- 
sively by f(0) = f(1) = 1 and forn = 1,2,... 

a) fa+)D= fa)—fa-—)). 

b) fa@+)D=fmfa-D. 

c) fat) =f)’ + fa-1?. 

d) fx+1)= f/f — 1). 


a valid recursive definition of a function f from the set 

of nonnegative integers to the set of integers. If f is well 

defined, find a formula for f (7) when n is a nonnegative 

integer and prove that your formula is valid. 

a) f(0) =0, f(x) =2f(n—2) forn > 1 

b) f0) =1, f@ = fm—1)—-1forn>1 

ec) fO=2, fI=3, fm=fa@—1)-1 for 
n>2 

d) f(0) =1, fC) =2, f(r) =2f(m — 2) forn > 2 

e) f(0)=1, f(x) =3f(— 1) if n is odd andn > 1 
and f(n) = 9f(n — 2) ifn is even andn > 2 


. Determine whether each of these proposed definitions is 


a valid recursive definition of a function f from the set 

of nonnegative integers to the set of integers. If f is well 

defined, find a formula for f (1) when n is a nonnegative 

integer and prove that your formula is valid. 

a) f(0)=1, fm) =—f(@-—1) forn> 1 

b) fO=1, fA) =0, f2)=2, f@) =2f( — 3) 
forn > 3 

ec) f() =0, fd) = 1, f@~) =2f(n+ 1) forn > 2 

d) f(0) =0, fQ) =1, f@) =2f(— 1) forn > 1 

e) f(0) =2, fm) = ft — 1) ifnisoddandn > 1 and 
f(a) =2f(n—2)ifn >2 
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7. Give a recursive definition of the sequence {a,}, n = 
1223, ct 
a) dy, = 6n. b) a, = 2n+1. 
c) ad, = 10". d) aq =5. 

8. Give a recursive definition of the sequence {a,}, n = 
1233 yt 
a) dy, =4n—2. b) a, =1+4+(-1)”. 


10. 


11. 


C) dh =n(n+ 1). d) a, = n. 


. Let F be the function such that F (n) is the sum of the first 


n positive integers. Give a recursive definition of F(n). 
Give a recursive definition of $,,(”), the sum of the inte- 
ger m and the nonnegative integer n. 


Give a recursive definition of P,,(”), the product of the 
integer m and the nonnegative integer n. 


In Exercises 12-19 f, is the nth Fibonacci number. 


12. 


13. 


*14, 


*15, 


*16. 


17. 


18. 


19. 


*20. 


*21. 


22. 


Prove that ch + fe peer t+ 2 = fnfnt1 when n is a 
positive integer. 

Prove that f) + f3 +---+ fon—1 = fon when n is a pos- 
itive integer. 

Show that fn41fn—1 — £2 = (—1)” when 2 is a positive 
integer. 


Show that fo fi + fif2 +--+ + fon-1 fon = f3, whenn 
is a positive integer. 
Show that fo-—fit fa—---— fon-1 t+ fon = 


fon—1 — 1 when n is a positive integer. 

Determine the number of divisions used by the Euclidean 
algorithm to find the greatest common divisor of the Fi- 
bonacci numbers f;, and f,+1, where n is a nonnegative 
integer. Verify your answer using mathematical induction. 


Let 
1 1 
a=|y a 
Show that 
Sn+1 fi 
A” = 
| Sn Sn-1 


when n is a positive integer. 


By taking determinants of both sides of the equation in 
Exercise 18, prove the identity given in Exercise 14. (Re- 


call that the determinant of the matrix e isad — bc.) 


Give a recursive definition of the functions max and 
min so that max(a), a2,..., @) and min(ay, a2, ..., ay) 
are the maximum and minimum of the n numbers 
a1, Q2,..., Gn, respectively. 

Let aj, da2,...,@,, and bj, bo,..., by be real numbers. 
Use the recursive definitions that you gave in Exercise 20 
to prove these. 


a) max(—aj, —a2,..., —dyn) = — min(dj, a2,..., dn) 
b) max(q; + bj, a. + b2,...,a, + bn) 

< max(d], d2,..., 4) + max(by, bo, ..., bn) 
ce) min(a; + b},a2 +bo,...,d, + bn) 

> min(a), a2,..., An) + min(d,, bo, ..., by) 


Show that the set S defined by 1 € S ands +t € S when- 
ever s € S andt € Sis the set of positive integers. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


Give a recursive definition of the set of positive integers 
that are multiples of 5. 

Give a recursive definition of 

a) the set of odd positive integers. 

b) the set of positive integer powers of 3. 

c) the set of polynomials with integer coefficients. 
Give a recursive definition of 

a) the set of even integers. 

b) the set of positive integers congruent to 2 modulo 3. 
c) the set of positive integers not divisible by 5. 

Let S be the subset of the set of ordered pairs of integers 
defined recursively by 


Basis step: (0,0) € S. 


Recursive step: If (a,b) € S, then (a+2,b+3) eS 

and (a+3,b+2)€S. 

a) List the elements of S produced by the first five ap- 
plications of the recursive definition. 

b) Use strong induction on the number of applications 
of the recursive step of the definition to show that 
5|a+b when (a,b) eS. 

c) Use structural induction to show that 5| a+b when 
(a,b) éES. 

Let S be the subset of the set of ordered pairs of integers 

defined recursively by 


Basis step: (0,0) € S. 


Recursive step: If (a,b) €S, then (a,b+1) €S, 

(a+1,b+1)€S,and(a+2,b+1)€S. 

a) List the elements of S produced by the first four ap- 
plications of the recursive definition. 

b) Use strong induction on the number of applications of 
the recursive step of the definition to show thata < 2b 
whenever (a, b) € S. 

c) Use structural induction to show that a < 2b when- 
ever (a,b) € S. 

Give a recursive definition of each of these sets of ordered 

pairs of positive integers. [Hint: Plot the points in the set 

in the plane and look for lines containing points in the 
set.] 

a) S={(a,b)|a€Z",beZ", anda + bis odd} 

b) S={(a,b)|a€Z*,b€ Z*, anda|b} 

ce) S={(a,b)|aeZ',beZ,and3|a+b} 

Give a recursive definition of each of these sets of or- 

dered pairs of positive integers. Use structural induction 

to prove that the recursive definition you found is correct. 

[Hint: To find a recursive definition, plot the points in the 

set in the plane and look for patterns.] 

a) S={(a,b)|a € Z*,b € Z*, anda +b is even} 

b) S = {(a,b)|a € Z*, b € Z*, anda or b is odd} 

ce) S={(a,b)|a € Zt, b € Z*,a + bis odd, and 3 | b} 

Prove that in a bit string, the string 01 occurs at most one 

more time than the string 10. 


Define well-formed formulae of sets, variables represent- 
ing sets, and operators from {~, U,M, —}. 


32. a) Give a recursive definition of the function ones(s), 
which counts the number of ones in a bit string s. 

b) Use structural induction to prove that ones(st) = 
ones(s) + ones(t). 

33. a) Give arecursive definition of the function m(s), which 
equals the smallest digit in a nonempty string of dec- 
imal digits. 

b) Use structural induction to prove that m(st) = 
min(m(s), m(t)). 

The reversal of a string is the string consisting of the symbols 

of the string in reverse order. The reversal of the string w is 

denoted by w’. 

34, Find the reversal of the following bit strings. 

a) 0101 b) 11011 ec) 1000 1001 0111 

35. Give a recursive definition of the reversal of a string. 

(Hint: First define the reversal of the empty string. Then 
write a string w of length n + 1 as xy, where x is a string 
of length n, and express the reversal of w in terms of x* 
and y.] 


* 36. Use structural induction to prove that (w;w2) Re wk wr. 


37. Give a recursive definition of w’, where w is a string and 
i is a nonnegative integer. (Here w! represents the con- 
catenation of i copies of the string w.) 
* 38. Give a recursive definition of the set of bit strings that are 
palindromes. 
39. When does a string belong to the set A of bit strings de- 
fined recursively by 


AEA 
Oxle Aifxe A, 


where A is the empty string? 


*40. Recursively define the set of bit strings that have more 
zeros than ones. 


41. Use Exercise 37 and mathematical induction to show that 
I(w') =i - 1(w), where w is a string and i is a nonnegative 
integer. 

* 42. Show that (w*)! = (w')® whenever w is a string and i is 
a nonnegative integer; that is, show that the ith power of 
the reversal of a string is the reversal of the ith power of 
the string. 


43. Use structural induction to show that n(T) > 2h(T) + 1, 
where T is a full binary tree, n(7) equals the number of 
vertices of T, and h(T) is the height of T. 


The set of leaves and the set of internal vertices of a full binary 
tree can be defined recursively. 


Basis step: The root r is a leaf of the full binary tree with 
exactly one vertex r. This tree has no internal vertices. 


Recursive step: The set of leaves of the tree T = T - T> is 

the union of the sets of leaves of 7; and of T>. The inter- 

nal vertices of T are the root r of T and the union of the 

set of internal vertices of 7; and the set of internal vertices 

of T>. 

44. Use structural induction to show that /(T), the number 
of leaves of a full binary tree T, is 1 more than i(7), the 
number of internal vertices of T. 
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45. Use generalized induction as was done in Example 13 to 
show that if a@),,, is defined recursively by ao,9 = 0 and 


— J4m—-1n+1 ifn =Oandm > 0 
t= | gen EL ifn = 0; 


then dm.,7 = m+n forall (m,n) Ee NXxN. 


46. Use generalized induction as was done in Example 13 to 
show that if a», is defined recursively by a,,; = 5 and 


— Jam-1yn +2 ifn =landm> 1 
Gm,n Gmn-1 +2 ifn >I, 


then dm. = 2(m +n) + 1 forall (m,n) € Z* x Zr. 


*47, A partition of a positive integer n is a way to write n 
as a sum of positive integers where the order of terms in 
the sum does not matter. For instance, 7 = 3+2+1+41 
is a partition of 7. Let P,,, equal the number of different 
partitions of m, and let Pi.» be the number of different 
ways to express m as the sum of positive integers not 
exceeding n. 


a) Show that Prim = Pn. 
b) Show that the following recursive definition for Py,» 


is correct: 
1 ifm= 1 
1 ifn=1 
Pinn => Pim ifm <n 
1+ Pnim-1 ifm=n>1 


Pran-1 t+ Pn-nn wim >n> 1. 


c) Find the number of partitions of 5 and of 6 using this 
recursive definition. 


~~ Consider an inductive definition of a version of Ackermann’s 


function. This function was named after Wilhelm Ackermann, 
a German mathematician who was a student of the great math- 
ematician David Hilbert. Ackermann’s function plays an im- 
portant role in the theory of recursive functions and in the study 
of the complexity of certain algorithms involving set unions. 
(There are several different variants of this function. All are 
called Ackermann’s function and have similar properties even 
though their values do not always agree.) 


2n fn =O 
A(m,n) = ifm = Vanda =0 
2 ifm >landn=1 


A(m—1,A(m,n—1)) ifm>landn>2 

Exercises 48-55 involve this version of Ackermann’s func- 
tion. 
48. Find these values of Ackermann’s function. 

a) A(1, 0) b) A(O, 1) 

c) AC, 1) d) A(2, 2) 
49. Show that A(m, 2) = 4 whenever m > 1. 
50. Show that A(1,) = 2” whenever n > 1. 
51. Find these values of Ackermann’s function. 

a) A(2, 3) *b) A(3, 3) 

*52, Find A(3, 4). 
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** 53, Prove that A(m, n + 1) > A(m,n) whenever m and n are 
nonnegative integers. 

*54, Prove that A(m + 1,n) > A(m, n) whenever m and n are 
nonnegative integers. 

55. Prove that A(i, 7) > j whenever i and j are nonnegative 
integers. 

56. Use mathematical induction to prove that a function F 
defined by specifying F(0) and a rule for obtaining 
F(n+ 1) from F(n) is well defined. 

57. Use strong induction to prove that a function F defined by 
specifying F'(0) and a rule for obtaining F(n + 1) from 
the values F(k) fork = 0, 1, 2,...,n 1s well defined. 

58. Show that each of these proposed recursive definitions of 
a function on the set of positive integers does not produce 
a well-defined function. 

a) F(n) =1+4+ F((n/2]) forn > 1 and F(1) = 1. 
b) Fm) =1+ F(n—3) for n>2, F(1) =2, and 


F(2) =3. 
c) F(n)=1+F(n/2) for n>2, F(1)=1, and 
FQ)=2, 


d) F(x) = 1+ F(n/2) if n is even and n > 2, F(n) = 
1 — F(n — 1) ifn is odd, and F(1) = 1. 
e) F(n) = 1+ F(n/2) if n is even and n > 2, F(n) = 
F(3n — 1) if nis odd andn > 3, and F(1) = 1. 
59. Show that each of these proposed recursive definitions of 
a function on the set of positive integers does not produce 
a well-defined function. 
a) Fm) =14+F(Lm+4+1)/2]) for n>1 = and 
FU) =1. 

b) F@) =1+ F(n — 2) forn > 2 and F(1) = 0. 

ce) Fn) = 1+ F(n/3) forn > 3, FO) = 1, FQ) = 2, 
and F'(3) = 3. 

d) F(n) = 1+ F(n/2) if n is even andn > 2, F(n) = 
1+ F(n — 2) ifn is odd, and F(1) = 1. 

e) Fn) =1+4+ F(F(n — 1)) ifn > 2 and F(1) = 2. 
Exercises 60-62 deal with iterations of the logarithm function. 
Let log n denote the logarithm of n to the base 2, as usual. The 
function log n is defined recursively by 
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n ifk =0 
iggy = log(log*—) n) if log*— n is defined 
and positive 


undefined otherwise. 


The iterated logarithm is the function log* n whose value atn 
is the smallest nonnegative integer k such that log n < 1. 
60. Find these values. 

a) log 16 b) log? 256 

©) log®) 265536 d) log 2 
61. Find the value of log* n for these values of n. 

a) 2 b) 4 c)8 d) 16 

e) 256 =f) 65536 —g) 27048 
62. Find the largest integer n such that log* n = 5. Determine 

the number of decimal digits in this number. 
Exercises 63-65 deal with values of iterated functions. Sup- 
pose that f(n) is a function from the set of real numbers, or 
positive real numbers, or some other set of real numbers, to 
the set of real numbers such that f(7) is monotonically in- 
creasing [that is, f(m) < f(m) when n <m) and f(n) <n 
for all n in the domain of f.] The function f )(n) is defined 
recursively by 


536 


n ifk =0 
fO@ = er 
SS (n)) ifk>0. 

Furthermore, let c be a positive real number. The iterated 

function /* is the number of iterations of f required to reduce 

its argument to c or less, so f*(n) is the smallest nonnegative 

integer k such that f*(n) < c. 

63. Let f(n) =n —a, where a is a positive integer. Find a 
formula for f (*) (7), What is the value of f5(n) when n 
is a positive integer? 

64. Let f(n) = n/2. Find a formula for f“ (n). What is the 
value of f{(n) when n is a positive integer? 


65. Let f(n) = Jn. Find a formula for f (n). What is the 
value of f3(n) when n is a positive integer? 


Introduction 


Sometimes we can reduce the solution to a problem with a particular set of input values to the 
solution of the same problem with smaller input values. For instance, the problem of finding 
the greatest common divisor of two positive integers a and b, where b > a, can be reduced 
to finding the greatest common divisor of a pair of smaller integers, namely, b moda and 
a, because gcd(b mod a, a) = gcd(a, b). When such a reduction can be done, the solution to 
the original problem can be found with a sequence of reductions, until the problem has been 
reduced to some initial case for which the solution is known. For instance, for finding the greatest 
common divisor, the reduction continues until the smaller of the two numbers is zero, because 
gcd(a, 0) = a whena > 0. 

We will see that algorithms that successively reduce a problem to the same problem with 
smaller input are used to solve a wide variety of problems. 


Here’s a famous 
humorous quote: “To 
understand recursion, you 
must first understand 
recursion.” 


DEFINITION 1 


Links 
EXAMPLE 1 


Extra p> 
Examples 2 


EXAMPLE 2 
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An algorithm is called recursive if it solves a problem by reducing it to an instance of the 
same problem with smaller input. 


We will describe a variety of different recursive algorithms in this section. 
Give a recursive algorithm for computing n!, where n is a nonnegative integer. 


Solution: We can build a recursive algorithm that finds n!, where n is a nonnegative integer, 
based on the recursive definition of n!, which specifies that n! = n - (n — 1)! when zn is a positive 
integer, and that 0! = 1. To find n! for a particular integer, we use the recursive step n times, 
each time replacing a value of the factorial function with the value of the factorial function at 
the next smaller integer. At this last step, we insert the value of 0!. The recursive algorithm we 
obtain is displayed as Algorithm 1. 

To help understand how this algorithm works, we trace the steps used by the algorithm 
to compute 4!. First, we use the recursive step to write 4! = 4. 3!. We then use the recursive 
step repeatedly to write 3! = 3-2!, 2! =2- I!, and 1! = 1 - O!. Inserting the value of 0! = 1, 
and working back through the steps, we see that 1! = 1-1=1,2!=2-1!=2,3!=3.2!= 
3-2=6,and4!=4-3!=4-6= 24. < 


ALGORITHM 1 A Recursive Algorithm for Computing n!. 


procedure factorial(n: nonnegative integer) 
ifn = O then return | 

else return n - factorial(n — 1) 

{output is n!} 


Example 2 shows how a recursive algorithm can be constructed to evaluate a function from its 
recursive definition. 


Give a recursive algorithm for computing a”, where a is a nonzero real number and n is a 
nonnegative integer. 


Solution: We can base a recursive algorithm on the recursive definition of a”. This definition 
states that a”+! = a-a" for n > 0 and the initial condition a° = 1. To find a”, successively 
use the recursive step to reduce the exponent until it becomes zero. We give this procedure in 
Algorithm 2. < 


ALGORITHM 2 A Recursive Algorithm for Computing a”. 


procedure power(a: nonzero real number, n: nonnegative integer) 
ifn = O then return 1| 

else return a - power(a,n — 1) 

{output is a”} 


Next we give a recursive algorithm for finding greatest common divisors. 
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Give a recursive algorithm for computing the greatest common divisor of two nonnegative 
integers a and b witha < b. 


Solution: We can base a recursive algorithm on the reduction gcd(a, b) = ged(b mod a, a) and 
the condition gcd(0, b) = b when b > 0. This produces the procedure in Algorithm 3, which is 
a recursive version of the Euclidean algorithm. 

We illustrate the workings of Algorithm 3 with a trace when the input is a = 5, b = 8. With 
this input, the algorithm uses the “else” clause to find that gcd(5, 8) = gcd(8 mod 5,5) = 
gcd(3, 5). It uses this clause again to find that gcd(3, 5) = gcd(5 mod 3, 3) = gcd(2, 3), then 
to get gcd(2, 3) = ged(3 mod 2, 2) = gcd(1, 2), then to get gcd(1, 2) = ged(2 mod 1, 1) = 
gcd(0, 1). Finally, to find gcd(0, 1) it uses the first step with a = 0 to find that gcd(0, 1) = 1. 
Consequently, the algorithm finds that gcd(5, 8) = 1. >| 


ALGORITHM 3 A Recursive Algorithm for Computing gcd(a, b). 


procedure gcd(a, b: nonnegative integers with a < b) 
if a = 0 then return b 

else return gcd(b mod a, a) 

{output is gcd(a, b)} 


Devise a recursive algorithm for computing b” mod m, where b, n, and m are integers with 
m>2,n>0, andl <b<~m. 


Solution: We can base a recursive algorithm on the fact that 
b" mod m = (b- (b"~! mod m)) mod m, 


which follows by Corollary 2 in Section 4.1, and the initial condition b° mod m = 1. We leave 
this as Exercise 12 for the reader. 

However, we can devise a much more efficient recursive algorithm based on the observation 
that 


b"” mod m = (b"/? mod m)* mod m 
when n is even and 
b" modm = ((pl/2 mod m)* mod m - b mod m) mod m 


when n is odd, which we describe in pseudocode as Algorithm 4. 

We trace the execution of Algorithm 4 with input b = 2,n = 5, andm = 3 to illustrate how 
it works. First, because n = 5 is odd we use the “else” clause to see that mpower(2, 5, 3) = 
(mpower(2, 2, 3)? mod 3 - 2 mod 3) mod 3. We next use the “else if” clause to see that 
mpower(2, 2, 3) = mpower(2, 1, 3)2 mod 3. Using the “else” clause again, we see that 
mpower(2, 1, 3) = (mpower(2, 0, 3)* mod 3 - 2 mod 3) mod 3. Finally, using the “if” clause, 
we see that mpower(2, 0, 3) = 1. Working backwards, it follows that mpower(2, 1, 3) = 
(17 mod 3 - 2 mod 3) mod 3 = 2, so mpower(2, 2, 3) = 2” mod 3 = 1, and finally 
mpower(2, 5, 3) = (1? mod 3-2 mod 3) mod 3 = 2. < 


EXAMPLE 5 


EXAMPLE 6 
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ALGORITHM 4 Recursive Modular Exponentiation. 


procedure mpower(b, n,m: integers with b > 0 andm > 2,n > 0) 
if n = 0 then 

return | 
else if 1 is even then 

return mpower(b, n/2,m)* mod m 
else 

return (mpower(b, |n/2), m)* mod m- b mod m) mod m 
{output is b” mod m} 


We will now give recursive versions of searching algorithms that were introduced in Section 3.1. 


Express the linear search algorithm as a recursive procedure. 


Solution: To search for the first occurrence of x in the sequence a, a2, ..., Gn, at the ith step of 
the algorithm, x and a; are compared. If x equals a;, then the algorithm returns 7, the location 
of x in the sequence. Otherwise, the search for the first occurrence of x is reduced to a search in 
a sequence with one fewer element, namely, the sequence aj+,..., d,. The algorithm returns 
0 when x is never found in the sequence after all terms have been examined. We can now give 
a recursive procedure, which is displayed as pseudocode in Algorithm 5. 

Let search (i, j, x) be the procedure that searches for the first occurrence of x in the sequence 
aj, aj41,..., aj. The input to the procedure consists of the triple (1,7, x). The algorithm 
terminates at a step if the first term of the remaining sequence is x or if there is only one term of 
the sequence and this is not x. If x is not the first term and there are additional terms, the same 
procedure is carried out but with a search sequence of one fewer term, obtained by deleting the 
first term of the search sequence. If the algorithm terminates without x having been found, the 
algorithm returns the value 0. < 


ALGORITHM 5 A Recursive Linear Search Algorithm. 


procedure search(i, j, x: i, j7,x integers, 1 <i < j <n) 
if a; = x then 
return i 
else if i = 7 then 
return 0 
else 
return search(i + 1, j, x) 
{output is the location of x in a1, a2, ..., A, if it appears; otherwise it is 0} 


Construct a recursive version of a binary search algorithm. 


Solution: Suppose we want to locate x in the sequence aj, a2, ..., dy of integers in increasing 
order. To perform a binary search, we begin by comparing x with the middle term, a) (n+1)/2). 
Our algorithm will terminate if x equals this term and return the location of this term in the 
sequence. Otherwise, we reduce the search to a smaller search sequence, namely, the first half 
of the sequence if x is smaller than the middle term of the original sequence, and the second 
half otherwise. We have reduced the solution of the search problem to the solution of the same 
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problem with a sequence at most half as long. If have we never encountered the search term x, 
our algorithm returns the value 0. We express this recursive version of a binary search algorithm 
as Algorithm 6. < 


ALGORITHM 6 A Recursive Binary Search Algorithm. 


procedure binary search(i, j, x: i, j,x integers, | <i < j <n) 
m:= |i + j)/21 
if x = a» then 
return m 
else if (x < ay andi < m) then 
return binary search(i,m — 1, x) 
else if (x > a, and j > m) then 
return binary search(m + 1, j, x) 
else return 0 
{output is location of x in a, a2, ..., Ay if it appears; otherwise it is 0} 


Proving Recursive Algorithms Correct 


Mathematical induction, and its variant strong induction, can be used to prove that a recursive 
algorithm is correct, that is, that it produces the desired output for all possible input values. 
Examples 7 and 8 illustrate how mathematical induction or strong induction can be used to 
prove that recursive algorithms are correct. First, we will show that Algorithm 2 is correct. 


Prove that Algorithm 2, which computes powers of real numbers, is correct. 


Solution: We use mathematical induction on the exponent n. 


BASIS STEP: If n =0, the first step of the algorithm tells us that power (a, 0) = 1. This is 
correct because a? = | for every nonzero real number a. This completes the basis step. 


INDUCTIVE STEP: The inductive hypothesis is the statement that power (a, k) = ak for all 
a # 0 for an arbitrary nonnegative integer k. That is, the inductive hypothesis is the statement 
that the algorithm correctly computes a‘. To complete the inductive step, we show that if the 
inductive hypothesis is true, then the algorithm correctly computes a‘t+!. Because k + 1 is a 
positive integer, when the algorithm computes a‘*!, the algorithm sets power (a, k + 1) = 
a: power (a, k). By the inductive hypothesis, we have power (a, k) = a‘, so power (a, k + 1) = 
a- power (a, k) = a-a* = a**!. This completes the inductive step. 

We have completed the basis step and the inductive step, so we can conclude that Algorithm 
2 always computes a” correctly when a # 0 and n is a nonnegative integer. < 


Generally, we need to use strong induction to prove that recursive algorithms are correct, 
rather than just mathematical induction. Example 8 illustrates this; it shows how strong induction 
can be used to prove that Algorithm 4 is correct. 


Prove that Algorithm 4, which computes modular powers, is correct. 


Solution: We use strong induction on the exponent n. 


BASIS STEP: Let b be an integer and m an integer with m > 2. Whenn = 0, the algorithm sets 
mpower(b, n, m) equal to 1. This is correct because b° mod m = 1. The basis step is complete. 
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INDUCTIVE STEP: For the inductive hypothesis we assume that mpower(b, j,m) = 
b/ mod m for all integers 0 < j < k whenever dD is a positive integer and m is an integer with 
m > 2. To complete the inductive step, we show that if the inductive hypothesis is correct, then 
mpower(b, k, m) = b* mod m. Because the recursive algorithm handles odd and even values 
of k differently, we split the inductive step into two cases. 

When k is even, we have 


mpower(b, k, m) = (mpower(b, k/2, m))* mod m = (b‘/2 mod m)* mod m = b* mod m, 


where we have used the inductive hypothesis to replace mpower(b, k/2, m) by b‘/? mod m. 
When k is odd, we have 


mpower(b, k, m) = ((mpower(b, [k/2], m))° mod m-bmod m) mod m 


= ((plk/?! mod m)- mod m - b mod m) mod m 
= b*\*/21+! mod m=b* mod m, 

using Corollary 2 in Section 4.1, because 2[k/2] + 1 = 2(k — 1)/2+1=k when k is odd. 
Here we have used the inductive hypothesis to replace mpower(b, |k/2|, m) by b!*/*! mod m. 
This completes the inductive step. 


We have completed the basis step and the inductive step, so by strong induction we know 
that Algorithm 4 is correct. < 


Recursion and Iteration 


A recursive definition expresses the value of a function at a positive integer in terms of the 
values of the function at smaller integers. This means that we can devise a recursive algorithm to 
evaluate a recursively defined function at a positive integer. Instead of successively reducing the 
computation to the evaluation of the function at smaller integers, we can start with the value of the 
function at one or more integers, the base cases, and successively apply the recursive definition to 
find the values of the function at successive larger integers. Such a procedure is called iterative. 
Often an iterative approach for the evaluation of a recursively defined sequence requires much 
less computation than a procedure using recursion (unless special-purpose recursive machines 
are used). This is illustrated by the iterative and recursive procedures for finding the nth Fibonacci 
number. The recursive procedure is given first. 


ALGORITHM 7 A Recursive Algorithm for Fibonacci Numbers. 


procedure fibonacci(n: nonnegative integer) 

if n = 0 then return 0 

else if n = 1 then return | 

else return fibonacci(n — 1) + fibonacci(n — 2) 
{output is fibonacci(n) } 


When we use a recursive procedure to find f,, we first express fy as fn—1 + fn—2. Then we 
replace both of these Fibonacci numbers by the sum of two previous Fibonacci numbers, and 
so on. When f; or fo arises, it is replaced by its value. 

Note that at each stage of the recursion, until f; or fo is obtained, the number of Fibonacci 
numbers to be evaluated has doubled. For instance, when we find 4 using this recursive algo- 
rithm, we must carry out all the computations illustrated in the tree diagram in Figure 1. This 
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FIGURE 1 Evaluating f4 Recursively. 


tree consists of a root labeled with f4, and branches from the root to vertices labeled with the 
two Fibonacci numbers f3 and f2 that occur in the reduction of the computation of f4. Each 
subsequent reduction produces two branches in the tree. This branching ends when fo and fj 
are reached. The reader can verify that this algorithm requires fn+1 — 1 additions to find fy. 

Now consider the amount of computation required to find /f;, using the iterative approach 
in Algorithm 8. 


ALGORITHM 8 An Iterative Algorithm for Computing Fibonacci Numbers. 


procedure iterative fibonacci(n: nonnegative integer) 
if n = 0 then return 0 


else 

x:=0 

y= 1 

fori :=1lton—-1 
Zie=xt+y 
xi=y 
yr Zz 

return y 


{output is the nth Fibonacci number} 


This procedure initializes x as fo = O and y as f; = 1. When the loop is traversed, the sum of x 
and y is assigned to the auxiliary variable z. Then x is assigned the value of y and y is assigned 
the value of the auxiliary variable z. Therefore, after going through the loop the first time, it 
follows that x equals f; and y equals fo + f; = fo. Furthermore, after going through the loop 
n — | times, x equals f,—1 and y equals f;, (the reader should verify this statement). Only n — 1 
additions have been used to find f,, with this iterative approach when n > 1. Consequently, this 
algorithm requires far less computation than does the recursive algorithm. 

We have shown that a recursive algorithm may require far more computation than an iterative 
one when arecursively defined function is evaluated. It is sometimes preferable to use a recursive 
procedure even if it is less efficient than the iterative procedure. In particular, this is true when 
the recursive approach is easily implemented and the iterative approach is not. (Also, machines 
designed to handle recursion may be available that eliminate the advantage of using iteration.) 
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FIGURE 2. The Merge Sort of 8, 2, 4, 6, 9, 7, 10, 1, 5, 3. 


The Merge Sort 


We now describe a recursive sorting algorithm called the merge sort algorithm. We will demon- 
strate how the merge sort algorithm works with an example before describing it in generality. 


Use the merge sort to put the terms of the list 8, 2, 4, 6, 9, 7, 10, 1, 5, 3 in increasing order. 


Solution: A merge sort begins by splitting the list into individual elements by successively 
splitting lists in two. The progression of sublists for this example is represented with the balanced 
binary tree of height 4 shown in the upper half of Figure 2. 

Sorting is done by successively merging pairs of lists. At the first stage, pairs of individual 
elements are merged into lists of length two in increasing order. Then successive merges of 
pairs of lists are performed until the entire list is put into increasing order. The succession of 
merged lists in increasing order is represented by the balanced binary tree of height 4 shown 
in the lower half of Figure 2 (note that this tree is displayed “upside down’). < 


In general, a merge sort proceeds by iteratively splitting lists into two sublists of equal 
length (or where one sublist has one more element than the other) until each sublist contains one 
element. This succession of sublists can be represented by a balanced binary tree. The procedure 
continues by successively merging pairs of lists, where both lists are in increasing order, into a 
larger list with elements in increasing order, until the original list is put into increasing order. 
The succession of merged lists can be represented by a balanced binary tree. 

We can also describe the merge sort recursively. To do a merge sort, we split a list into 
two sublists of equal, or approximately equal, size, sorting each sublist using the merge sort 
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algorithm, and then merging the two lists. The recursive version of the merge sort is given in 
Algorithm 9. This algorithm uses the subroutine merge, which is described in Algorithm 10. 


ALGORITHM 9 A Recursive Merge Sort. 


procedure mergesort(L = a,,..., dy) 
ifn > 1 then 
m := |n/2] 
Ly := a1, 02,...,Am 
L2 := Am+1, Am42, +--+ 4n 
L := merge(mergesort(L,), mergesort(L2)) 
{L is now sorted into elements in nondecreasing order} 


An efficient algorithm for merging two ordered lists into a larger ordered list is needed to 
implement the merge sort. We will now describe such a procedure. 


Merge the two lists 2, 3, 5, 6 and 1, 4. 


Solution: Table 1 illustrates the steps we use. First, compare the smallest elements in the two 
lists, 2 and 1, respectively. Because | is the smaller, put it at the beginning of the merged list 
and remove it from the second list. At this stage, the first list is 2, 3, 5, 6, the second is 4, and 
the combined list is 1. 

Next, compare 2 and 4, the smallest elements of the two lists. Because 2 is the smaller, add 
it to the combined list and remove it from the first list. At this stage the first list is 3, 5, 6, the 
second is 4, and the combined list is 1, 2. 

Continue by comparing 3 and 4, the smallest elements of their respective lists. Because 3 
is the smaller of these two elements, add it to the combined list and remove it from the first list. 
At this stage the first list is 5, 6, and the second is 4. The combined list is 1, 2, 3. 

Then compare 5 and 4, the smallest elements in the two lists. Because 4 is the smaller of 
these two elements, add it to the combined list and remove it from the second list. At this stage 
the first list is 5, 6, the second list is empty, and the combined list is 1, 2, 3, 4. 

Finally, because the second list is empty, all elements of the first list can be appended to 
the end of the combined list in the order they occur in the first list. This produces the ordered 
list 1, 2, 3, 4, 5, 6. < 


We will now consider the general problem of merging two ordered lists L; and L2 into 
an ordered list L. We will describe an algorithm for solving this problem. Start with an empty 
list L. Compare the smallest elements of the two lists. Put the smaller of these two elements at 
the right end of L, and remove it from the list it was in. Next, if one of L; and L2 is empty, 
append the other (nonempty) list to L, which completes the merging. If neither L; nor Lo is 
empty, repeat this process. Algorithm 10 gives a pseudocode description of this procedure. 


TABLE 1 Merging the Two Sorted Lists 2, 3, 5, 6 and 1, 4. 


14 le2 
4 1 2<4 
4 12 3<4 
4 123 4<5 
1234 
123456 


LEMMA 1 
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We will need estimates for the number of comparisons used to merge two ordered lists in the 
analysis of the merge sort. We can easily obtain such an estimate for Algorithm 10. Each time 
a comparison of an element from L; and an element from L> is made, an additional element 
is added to the merged list L. However, when either L; or Lz is empty, no more comparisons 
are needed. Hence, Algorithm 10 is least efficient when m + n — 2 comparisons are carried out, 
where m and n are the number of elements in L; and Ly, respectively, leaving one element in 
each of L; and Lz. The next comparison will be the last one needed, because it will make one 
of these lists empty. Hence, Algorithm 10 uses no more than m + n — 1 comparisons. Lemma 1 
summarizes this estimate. 


ALGORITHM 10 Merging Two Lists. 


procedure merge(L,, L2: sorted lists) 
L := empty list 
while L; and L> are both nonempty 
remove smaller of first elements of L; and L2 from its list; put it at the right end of L 
if this removal makes one list empty then remove all elements from the other list and 
append them to L 
return L{L is the merged list with elements in increasing order} 


Two sorted lists with m elements and n elements can be merged into a sorted list using no 
more than m + n — | comparisons. 


Sometimes two sorted lists of length m and n can be merged using far fewer than m +n — 1 
comparisons. For instance, when m = 1, a binary search procedure can be applied to put the 
one element in the first list into the second list. This requires only [logn] comparisons, which 
is much smaller than m+n -— 1=n, for m = 1. On the other hand, for some values of m 
and n, Lemma | gives the best possible bound. That is, there are lists with m and n elements 
that cannot be merged using fewer than m + n — | comparisons. (See Exercise 47.) 

We can now analyze the complexity of the merge sort. Instead of studying the general 
problem, we will assume that n, the number of elements in the list, is a power of 2, say 2”. This 
will make the analysis less complicated, but when this is not the case, various modifications can 
be applied that will yield the same estimate. 

At the first stage of the splitting procedure, the list is split into two sublists, of 2’””~ * elements 
each, at level 1 of the tree generated by the splitting. This process continues, splitting the two 
sublists with 2’””—! elements into four sublists of 2’”~2 elements each at level 2, and so on. In 
general, there are 2k-1 lists at level k — 1, each with 2”—*+! elements. These lists at level k — 1 
are split into 2* lists at level k, each with 2””—* elements. At the end of this process, we have 2” 
lists each with one element at level m. 

We start merging by combining pairs of the 2” lists of one element into 2’"~! lists, at level 
m — 1, each with two elements. To do this, 2””~! pairs of lists with one element each are merged. 
The merger of each pair requires exactly one comparison. 

The procedure continues, so that at level k (k =m,m—1,m-—2,...,3,2, 1), 2* lists 
each with 2”—* elements are merged into 2‘-] lists, each with 2””—*+! elements, at level k — 1. 
To do this a total of 2‘~! mergers of two lists, each with 2m—-k elements, are needed. But, 
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by Lemma 1, each of these mergers can be carried out using at most 2”—* + 2”-* —] = 
2™-k+l _ 1 comparisons. Hence, going from level k to k — 1 can be accomplished using at 
most 2*-1(gm-F+! — 1) comparisons. 

Summing all these estimates shows that the number of comparisons required for the merge 
sort is at most 


m m m 


2 gk-lgm—kt! _ 4) — > aa >. 2k-1 = m2™ — (2 —1) =nlogn—n+1, 
k=1 k=1 k=1 


because m = logn and n = 2”. (We evaluated )77"_ , 2” by noting that it is the sum of m 
identical terms, each equal to 2”. We evaluated )7;'_ 2*-1 using the formula for the sum of 
the terms of a geometric progression from Theorem | of Section 2.4.) 

Theorem | summarizes what we have discovered about the worst-case complexity of the 
merge sort algorithm. 


The number of comparisons needed to merge sort a list with n elements is O(n logn). 


In Chapter 11 we will show that the fastest comparison-based sorting algorithm have 
O(n logn) time complexity. (A comparison-based sorting algorithm has the comparison of 
two elements as its basic operation.) Theorem | tells us that the merge sort achieves this best 
possible big-O estimate for the complexity of a sorting algorithm. We describe another efficient 
algorithm, the quick sort, in the preamble to Exercise 50. 


. Trace Algorithm | when it is given n = 5 as input. That 
is, show all steps used by Algorithm 1 to find 5!, as is 
done in Example | to find 4!. 


. Trace Algorithm | when it is given n = 6 as input. That 
is, show all steps used by Algorithm | to find 6!, as is 
done in Example | to find 4!. 


. Trace Algorithm 3 when it finds gcd(8, 13). That is, show 
all the steps used by Algorithm 3 to find gcd(8, 13). 


. Trace Algorithm 3 when it finds gcd(12, 17). That is, 
show all the steps used by Algorithm 3 to find gcd(12, 17). 


. Trace Algorithm 4 when it is given m = 5, n = 11, and 
b = 3 as input. That is, show all the steps Algorithm 4 
uses to find 3!!mod 5. 


. Trace Algorithm 4 when it is given m = 7, n = 10, and 
b = 2 as input. That is, show all the steps Algorithm 4 
uses to find 2!°mod 7. 


. Give a recursive algorithm for computing nx whenever n 
is a positive integer and x is an integer, using just addition. 


. Give a recursive algorithm for finding the sum of the 
first n positive integers. 


. Give a recursive algorithm for finding the sum of the 
first n odd positive integers. 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


Give a recursive algorithm for finding the maximum of 
a finite set of integers, making use of the fact that the 
maximum of n integers is the larger of the last integer in 
the list and the maximum of the first n — | integers in the 
list. 


Give a recursive algorithm for finding the minimum of a 
finite set of integers, making use of the fact that the min- 
imum of n integers is the smaller of the last integer in the 
list and the minimum of the first n — 1 integers in the list. 


Devise a recursive algorithm for finding x” mod m when- 
ever n, x, and m are positive integers based on the fact 
that x” mod m = (x”—! mod m - x mod m) mod m. 


Give a recursive algorithm for finding n! mod m when- 
ever n and m are positive integers. 


Give a recursive algorithm for finding a mode of a list of 
integers. (A mode is an element in the list that occurs at 
least as often as every other element.) 


Devise a recursive algorithm for computing the greatest 
common divisor of two nonnegative integers a and b with 
a < busing the fact that gcd(a, b) = gcd(a, b — a). 


Prove that the recursive algorithm for finding the sum of 
the first n positive integers you found in Exercise 8 is 
correct. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


* 26. 


* 27, 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


37. 


Describe a recursive algorithm for multiplying two non- 
negative integers x and y based on the fact that xy = 
2(x - (y/2)) when y is even and xy = 2(x- |y/2]) +x 
when y is odd, together with the initial condition xy = 0 
when y = 0. 

Prove that Algorithm 1 for computing n! when n is a non- 
negative integer is correct. 

Prove that Algorithm 3 for computing gcd(a, b) when a 
and b are positive integers with a < b is correct. 

Prove that the algorithm you devised in Exercise 17 is 
correct. 

Prove that the recursive algorithm that you found in Ex- 
ercise 7 is correct. 

Prove that the recursive algorithm that you found in Ex- 
ercise 10 is correct. 

Devise a recursive algorithm for computing n? where n 
is a nonnegative integer, using the fact that (n + 1)? = 
n? + 2n + 1. Then prove that this algorithm is correct. 
Devise a recursive algorithm to find a’, where a is a 
real number and n is a positive integer. [Hint: Use the 
equality i (a?’)?] 

How does the number of multiplications used by the al- 
gorithm in Exercise 24 compare to the number of multi- 
plications used by Algorithm 2 to evaluate a?" ? 

Use the algorithm in Exercise 24 to devise an algo- 
rithm for evaluating a” when n is a nonnegative integer. 
[Hint: Use the binary expansion of n.] 

How does the number of multiplications used by the al- 
gorithm in Exercise 26 compare to the number of multi- 
plications used by Algorithm 2 to evaluate a”? 

How many additions are used by the recursive and itera- 
tive algorithms given in Algorithms 7 and 8, respectively, 
to find the Fibonacci number /f7? 

Devise a recursive algorithm to find the nth term of the se- 
quence defined by ap = 1, a) = 2, and ay = an—1 - Gn-2, 
forn = 2,3,4,.... 

Devise an iterative algorithm to find the nth term of the 
sequence defined in Exercise 29. 

Is the recursive or the iterative algorithm for finding the 
sequence in Exercise 29 more efficient? 

Devise a recursive algorithm to find the nth term of 
the sequence defined by ap = 1, aj = 2, az = 3, and 
Qn = An—1 + n—2 + Gn_3, forn = 3,4,5,.... 

Devise an iterative algorithm to find the nth term of the 
sequence defined in Exercise 32. 

Is the recursive or the iterative algorithm for finding the 
sequence in Exercise 32 more efficient? 

Give iterative and recursive algorithms for finding the nth 
term of the sequence defined by ag = 1, ay = 3, a2 = 5, 
and dy = ayn_1 - ae : eas Which is more efficient? 
Give a recursive algorithm to find the number of parti- 
tions of a positive integer based on the recursive definition 
given in Exercise 47 in Section 5.3. 

Give a recursive algorithm for finding the reversal of a bit 
string. (See the definition of the reversal of a bit string in 
the preamble of Exercise 34 in Section 5.3.) 


38. 


39. 


40. 


*41, 


42. 


43. 


44. 


45. 


46. 


47. 


* 48, 


* 49, 


50. 
51. 


52. 
53. 


54. 


55. 
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Give a recursive algorithm for finding the string w’, the 
concatenation of i copies of w, when w is a bit string. 


Prove that the recursive algorithm for finding the reversal 
of a bit string that you gave in Exercise 37 is correct. 


Prove that the recursive algorithm for finding the concate- 
nation of i copies of a bit string that you gave in Exercise 
38 is correct. 


Give a recursive algorithm for tiling a 2” x 2” checker- 
board with one square missing using right triominoes. 


Give arecursive algorithm for triangulating a simple poly- 
gon with n sides, using Lemma | in Section 5.2. 


Give a recursive algorithm for computing values of the 
Ackermann function. [Hint: See the preamble to Exer- 
cise 48 in Section 5.3.] 


Use a merge sort to sort 4, 3, 2,5, 1, 8, 7, 6 into increasing 
order. Show all the steps used by the algorithm. 


Use a merge sort to sort b, d, a, fg, h, z, p, o, k into al- 
phabetic order. Show all the steps used by the algorithm. 


How many comparisons are required to merge these pairs 
of lists using Algorithm 10? 

a) 1,3,5, 7, 9; 2, 4, 6, 8, 10 

b) 1, 2,3, 4, 5; 6, 7, 8,9, 10 

c) 1,5, 6, 7, 8; 2, 3, 4, 9, 10 

Show that for all positive integers m and n there are sorted 
lists with m elements and n elements, respectively, such 
that Algorithm 10 uses m + n — 1 comparisons to merge 
them into one sorted list. 


What is the least number of comparisons needed to merge 
any two lists in increasing order into one list in increasing 
order when the number of elements in the two lists are 
a) 1,4? b) 2, 4? c) 3, 4? d) 4, 4? 

Prove that the merge sort algorithm is correct. 


The quick sort is an efficient algorithm. To sort 
a\,a2,...,d, this algorithm begins by taking the first 
element a; and forming two sublists, the first contain- 
ing those elements that are less than ay, in the order they 
arise, and the second containing those elements greater 
than a1, in the order they arise. Then aj is put at the end 
of the first sublist. This procedure is repeated recursively 
for each sublist, until all sublists contain one item. The or- 
dered list of n items is obtained by combining the sublists 
of one item in the order they occur. 

Sort 3, 5, 7, 8, 1, 9, 2, 4, 6 using the quick sort. 

Let a), d2,...,d, be a list of n distinct real numbers. 
How many comparisons are needed to form two sublists 
from this list, the first containing elements less than a1 
and the second containing elements greater than a,? 
Describe the quick sort algorithm using pseudocode. 
What is the largest number of comparisons needed to or- 
der a list of four elements using the quick sort algorithm? 
What is the least number of comparisons needed to order 
a list of four elements using the quick sort algorithm? 


Determine the worst-case complexity of the quick sort 
algorithm in terms of the number of comparisons used. 
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Program Correctness 


DEFINITION 1 


Links 


Introduction 


Suppose that we have designed an algorithm to solve a problem and have written a program 
to implement it. How can we be sure that the program always produces the correct answer? 
After all the bugs have been removed so that the syntax is correct, we can test the program with 
sample input. It is not correct if an incorrect result is produced for any sample input. But even if 
the program gives the correct answer for all sample input, it may not always produce the correct 
answer (unless all possible input has been tested). We need a proof to show that the program 
always gives the correct output. 

Program verification, the proof of correctness of programs, uses the rules of inference and 
proof techniques described in this chapter, including mathematical induction. Because an incor- 
rect program can lead to disastrous results, a large amount of methodology has been constructed 
for verifying programs. Efforts have been devoted to automating program verification so that it 
can be carried out using a computer. However, only limited progress has been made toward this 
goal. Indeed, some mathematicians and theoretical computer scientists argue that it will never 
be realistic to mechanize the proof of correctness of complex programs. 

Some of the concepts and methods used to prove that programs are correct will be introduced 
in this section. Many different methods have been devised for proving that programs are correct. 
We will discuss a widely used method for program verification introduced by Tony Hoare in 
this section; several other methods are also commonly used. Furthermore, we will not develop a 
complete methodology for program verification in this book. This section is meant to be a brief 
introduction to the area of program verification, which ties together the rules of logic, proof 
techniques, and the concept of an algorithm. 


Program Verification 


A program is said to be correct if it produces the correct output for every possible input. A proof 
that a program is correct consists of two parts. The first part shows that the correct answer is 
obtained if the program terminates. This part of the proof establishes the partial correctness 
of the program. The second part of the proof shows that the program always terminates. 

To specify what it means for a program to produce the correct output, two propositions 
are used. The first is the initial assertion, which gives the properties that the input values must 
have. The second is the final assertion, which gives the properties that the output of the program 
should have, if the program did what was intended. The appropriate initial and final assertions 
must be provided when a program is checked. 


A program, or program segment, S is said to be partially correct with respect to the 
initial assertion p and the final assertion g if whenever p is true for the input values 
of S and S terminates, then q is true for the output values of S. The notation p{S}q in- 
dicates that the program, or program segment, S is partially correct with respect to the initial 
assertion p and the final assertion q. 


Note: The notation p{S}g is known as a Hoare triple. Tony Hoare introduced the concept of 
partial correctness. 

Note that the notion of partial correctness has nothing to do with whether a program termi- 
nates; it focuses only on whether the program does what it is expected to do if it terminates. 

A simple example illustrates the concepts of initial and final assertions. 


EXAMPLE 1 


Extra 
Examples 


6 
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Show that the program segment 


is correct with respect to the initial assertion p: x = | and the final assertion q: z = 3. 


Solution: Suppose that p is true, so that x = | as the program begins. Then y is assigned the 
value 2, and z is assigned the sum of the values of x and y, which is 3. Hence, S is correct with 
respect to the initial assertion p and the final assertion g. Thus, p{S}q is true. < 


Rules of Inference 


A useful rule of inference proves that a program is correct by splitting the program into a 
sequence of subprograms and then showing that each subprogram is correct. 

Suppose that the program S is split into subprograms S and S2. Write S = S1; S2 to indicate 
that S is made up of S; followed by Sz. Suppose that the correctness of $, with respect to the 
initial assertion p and final assertion g, and the correctness of Sz with respect to the initial 
assertion q and the final assertion r, have been established. It follows that if p is true and Sj is 
executed and terminates, then g is true; and if qg is true, and Sz executes and terminates, then r 
is true. Thus, if p is true and S = $j; S2 is executed and terminates, then r is true. This rule of 
inference, called the composition rule, can be stated as 


Pi{Si}¢ 
q{S2}r 


J. p{Sq; So}r. 


This rule of inference will be used later in this section. 

Next, some rules of inference for program segments involving conditional statements and 
loops will be given. Because programs can be split into segments for proofs of correctness, this 
will let us verify many different programs. 


Conditional Statements 


First, rules of inference for conditional statements will be given. Suppose that a program 
segment has the form 


if condition then 
S 


where S is a block of statements. Then S is executed if condition 1s true, and it is not executed 
when condition is false. To verify that this segment is correct with respect to the initial assertion p 
and final assertion g, two things must be done. First, it must be shown that when p is true and 
condition is also true, then q is true after § terminates. Second, it must be shown that when p 
is true and condition is false, then q is true (because in this case S$ does not execute). 

This leads to the following rule of inference: 


(p A condition){S}q 
(p A —condition) > q 


.. p{if condition then S}q. 
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Example 2 illustrates how this rule of inference is used. 


EXAMPLE 2 Verify that the program segment 


ifx > y then 
yi=x 


is correct with respect to the initial assertion T and the final assertion y > x. 


Solution: When the initial assertion is true and x > y, the assignment y := x is carried out. 
Hence, the final assertion, which asserts that y > x, is true in this case. Moreover, when the 
initial assertion is true and x > y is false, so that x < y, the final assertion is again true. Hence, 
using the rule of inference for program segments of this type, this program is correct with respect 
to the given initial and final assertions. < 


Similarly, suppose that a program has a statement of the form 


if condition then 
Si 

else 
So 


If condition is true, then S; executes; if condition is false, then Sz executes. To verify that this 
program segment is correct with respect to the initial assertion p and the final assertion g, two 
things must be done. First, it must be shown that when p is true and condition is true, then q 
is true after S$; terminates. Second, it must be shown that when p is true and condition is false, 
then gq is true after S> terminates. This leads to the following rule of inference: 


(p A condition){S1}q 
(p A —condition){S2}q 


. p{if condition then Sj else S2}q. 


C. ANTHONY R. HOARE (BORN 1934) Tony Hoare was born in Colombo, Ceylon (now known as Sri 
Lanka), where his father was a civil servant of the British Empire and his mother’s father owned a plantation. 
He spent his early childhood in Ceylon, moving to England in 1945. Hoare studied philosophy, together with 
the classics, at the University of Oxford, where he became interested in computing as a result of his fascination 
with the power of mathematical logic and the certainty of mathematical truth. He received his bachelors degree 
from Oxford in 1956. 

Hoare learned Russian during his service in the Royal Navy, and latter studied the computer translation of 
natural languages at Moscow State University. He returned to England in 1960, taking a job at a small computer manufacturer, 
where he wrote a compiler for the programming language Algol. In 1968, he became Professor of Computing Science at the Queen’s 
University, Belfast; in 1977, he moved to the University of Oxford as Professor of Computing; he is now Professor Emeritus. He is a 
Fellow of the Royal Society and also holds a position at Microsoft Research in Cambridge. 

Hoare has made many contributions to the theory of programming languages and to programming methodology. He was first 
to define a programming language based on how programs could be proved correct with respect to their specifications. Hoare also 
invented quick sort, one of the most commonly used sorting algorithms (see the preamble to Exercise 50 in Section 5.4). He received 
the ACM Turing Award in 1980 and in 2000 he was knighted for services to education and computer science. Hoare is a noted writer 
in the technical and social aspects of computer science. 


EXAMPLE 3 


Links 


EXAMPLE 4 


Extra p> 
Examples 2 
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Example 3 illustrates how this rule of inference is used. 


Verify that the program segment 


if x < 0 then 
abs := —x 
else 
abs :=x 


is correct with respect to the initial assertion T and the final assertion abs = |x|. 


Solution: Two things must be demonstrated. First, it must be shown that if the initial assertion is 
true and x < 0, then abs = |x|. This is correct, because when x < 0 the assignment statement 
abs := —x sets abs = —x, which is |x| by definition when x < 0. Second, it must be shown 
that if the initial assertion is true and x < 0 is false, so that x > 0, then abs = |x|. This is also 
correct, because in this case the program uses the assignment statement abs := x, and x is |x| 
by definition when x > 0, so abs := x. Hence, using the rule of inference for program segments 
of this type, this segment is correct with respect to the given initial and final assertions. < 


Loop Invariants 


Next, proofs of correctness of while loops will be described. To develop a rule of inference for 
program segments of the type 


while condition 
S 


note that S is repeatedly executed until condition becomes false. An assertion that remains true 
each time S is executed must be chosen. Such an assertion is called a loop invariant. In other 
words, p is a loop invariant if (p A condition){S} p is true. 

Suppose that p is a loop invariant. It follows that if p is true before the program segment is 
executed, p and —condition are true after termination, if it occurs. This rule of inference is 


(p A condition){S} p 


.. p{while condition S}(— condition A p). 


The use of a loop invariant is illustrated in Example 4. 


A loop invariant is needed to verify that the program segment 


i:=1 

factorial := 1 

while i <n 
i:=i+l 


factorial := factorial «i 


terminates with factorial = n! when n is a positive integer. 
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EXAMPLE 5 


Let p be the assertion “factorial = i! andi <n.” We first prove that p is a loop invariant. 
Suppose that, at the beginning of one execution of the while loop, p is true and the condition of 
the while loop holds; in other words, assume that factorial = i! and that i < n. The new values 
inew and factorial,.,, of i and factorial are inew = i + 1 and factorial, = factorial - (i + 1) = 
(i + 1)! = inew!. Because i < n, we also have ineyw = i + 1 <n. Thus, p is true at the end of 
the execution of the loop. This shows that p is a loop invariant. 

Now we consider the program segment. Just before entering the loop, i = 1 <n and 
factorial = | = 1! =i! both hold, so p is true. Because p is a loop invariant, the rule of infer- 
ence just introduced implied that if the while loop terminates, it terminates with p true and with 
i <n false. In this case, at the end, factorial = i! andi <n are true, buti < n is false; in other 
words, i = n and factorial = i! = n!, as desired. 

Finally, we need to check that the while loop actually terminates. At the beginning of the 
program i is assigned the value 1, so after n — 1 traversals of the loop, the new value of i will 
be n, and the loop terminates at that point. < 


A final example will be given to show how the various rules of inference can be used to 
verify the correctness of a longer program. 


We will outline how to verify the correctness of the program S for computing the product of 
two integers. 


procedure multiply(m, n: integers) 


ifn < Othena := —n 
'Velsea:=n 


while k <a 
S3 x:=x+m 
k:=k+1 
ifn < 0 then product := —x 
4) else product := x 


return product 
{product equals mn} 


The goal is to prove that after S is executed, product has the value mn. The proof of correctness 
can be carried out by splitting S into four segments, with S = $1; $2; $3; S4, as shown in the 
listing of S. The rule of composition can be used to build the correctness proof. Here is how the 
argument proceeds. The details will be left as an exercise for the reader. 

Let p be the initial assertion “m and zn are integers.” Then, it can be shown that p{5S1}q is true, 
when q is the proposition p A (a = |n|). Next, let be the proposition g A (k = 0) A (x = 0). It 
is easily verified that g{S2}r is true. It can be shown that “x = mk and k < a” is an invariant for 
the loop in $3. Furthermore, it is easy to see that the loop terminates after a iterations, withk = a, 
so x = ma at this point. Because r implies that x = m - 0 and 0 < a, the loop invariant is true 
before the loop is entered. Because the loop terminates with k = a, it follows that r{S3}s is true 
where s is the proposition “x = ma anda = |n|.” Finally, it can be shown that Sq is correct with 
respect to the initial assertion s and final assertion t, where ¢ is the proposition “product = mn.” 

Putting all this together, because p{S,}g, g{S2}r, r{S3}s, and s{S4}t are all true, it fol- 
lows from the rule of composition that p{S}¢ is true. Furthermore, because all four segments 
terminate, S does terminate. This verifies the correctness of the program. 


Exercises 


1. Prove that the program segment 
y:=1 
Zi=xt+y 
is correct with respect to the initial assertion x = 0 and 
the final assertion z = 1. 
2. Verify that the program segment 
if x < 0 then x := 0 
is correct with respect to the initial assertion T and the 
final assertion x > 0. 
3. Verify that the program segment 
X= 2 
Zi=xty 
if y > 0 then 
z:=z+1 
else 
zi=0 
is correct with respect to the initial assertion y = 3 and 
the final assertion z = 6. 


4. Verify that the program segment 


if x < y then 
min := x 
else 
min := y 


is correct with respect to the initial assertion T and the 
final assertion (x < yA min =x)V (x > yAmin= y). 
*5, Devise a rule of inference for verification of partial cor- 
rectness of statements of the form 
if condition | then 
S} 
else if condition 2 then 
So 


else 
Sn 


where S}, S2,..., S, are blocks. 


6. Use the rule of inference developed in Exercise 5 to verify 
that the program 


Key Terms and Results 


TERMS 


sequence: a function with domain that is a subset of the set of 
integers 


geometric progression: a sequence of the forma, ar, ar’, sees 
where a and r are real numbers 


arithmetic progression: a sequence of the form a, a+d, 
a-+2d,..., where a and d are real numbers 
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if x < 0 then 
y = |x| /x 
else if x > 0 then 
y = Qlal/x 
else if x = 0 then 
y= 2 
is correct with respect to the initial assertion T and the 
final assertion y = 2. 

7. Use a loop invariant to prove that the following program 
segment for computing the nth power, where n is a posi- 
tive integer, of a real number x is correct. 

power := 1 

i:=1 

while i <n 
power := power * x 
i:=it+1 

*8. Prove that the iterative program for finding f, given in 
Section 5.4 is correct. 

9. Provide all the details in the proof of correctness given in 
Example 5. 

10. Suppose that both the conditional statement pp — p, and 
the program assertion p;{S}q are true. Show that po{S}q 
also must be true. 

11. Suppose that both the program assertion p{S}qo and 
the conditional statement gg — q, are true. Show that 
p{S}qi also must be true. 

12. This program computes quotients and remainders. 

ri=a 
q:=0 
while r > d 
ri=r-—d 
q:i=qtl 
Verify that it is partially correct with respect to the ini- 
tial assertion “a and d are positive integers” and the final 
assertion “gq and r are integers such that a = dq +r and 
O<r<d.” 

13. Use a loop invariant to verify that the Euclidean algorithm 
(Algorithm 1 in Section 4.3) is partially correct with re- 
spect to the initial assertion “‘a and b are positive integers” 
and the final assertion “x = gcd(a, b).” 


the principle of mathematical induction: the statement 
Vn P(n) is true if P(1) is true and VA[P(k) > P(k + 1)] 
is true. 


basis step: the proof of P(1) in a proof by mathematical in- 
duction of Vn P (n) 


inductive step: the proof of P(k) — P(k+ 1) for all pos- 
itive integers k in a proof by mathematical induction of 
Vn P(n) 
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strong induction: the statement Vn P (1) is true if P(1) is true 
and Vk[(P(1) A---A P(k)) > P(k + 1)] is true 

well-ordering property: Every nonempty set of nonnegative 
integers has a least element. 

recursive definition of a function: a definition of a function 
that specifies an initial set of values and a rule for obtaining 
values of this function at integers from its values at smaller 
integers 

recursive definition of a set: a definition of a set that specifies 
an initial set of elements in the set and a rule for obtaining 
other elements from those in the set 

structural induction: a technique for proving results about 
recursively defined sets 

recursive algorithm: an algorithm that proceeds by reducing 
a problem to the same problem with smaller input 


Review Questions 


1. a) Can you use the principle of mathematical induction 
to find a formula for the sum of the first n terms of a 
sequence? 

b) Can you use the principle of mathematical induction 
to determine whether a given formula for the sum of 
the first n terms of a sequence is correct? 

c) Find a formula for the sum of the first n even positive 
integers, and prove it using mathematical induction. 


2. a) For which positive integers n is 11n + 17 < 2”? 
b) Prove the conjecture you made in part (a) using math- 
ematical induction. 


3. a) Which amounts of postage can be formed using only 
5-cent and 9-cent stamps? 
b) Prove the conjecture you made using mathematical 
induction. 
c) Prove the conjecture you made using strong induction. 
d) Finda proof of your conjecture different from the ones 
you gave in (b) and (c). 
4. Give two different examples of proofs that use strong in- 
duction. 


5. a) State the well-ordering property for the set of positive 
integers. 

b) Use this property to show that every positive inte- 
ger greater than one can be written as the product of 
primes. 

6. a) Explain why a function f from the set of positive in- 
tegers to the set of real numbers is well-defined if it is 
defined recursively by specifying f(1) and a rule for 
finding f(n) from f(n — 1). 

b) Provide a recursive definition of the function f(n) = 
(n+ 1)!. 

7. a) Give arecursive definition of the Fibonacci numbers. 

b) Show that f, > a2 whenever n > 3, where Sn is 
the nth term of the Fibonacci sequence and a = 


(1+ /5)/2. 


merge sort: a sorting algorithm that sorts a list by splitting it 
in two, sorting each of the two resulting lists, and merging 
the results into a sorted list 


iteration: a procedure based on the repeated use of operations 
in a loop 


program correctness: verification that a procedure always 
produces the correct result 


loop invariant: a property that remains true during every 
traversal of a loop 


initial assertion: the statement specifying the properties of the 
input values of a program 


final assertion: the statement specifying the properties the out- 
put values should have if the program worked correctly 


8. a) Explain why a sequence ay is well defined if it is de- 
fined recursively by specifying a; and a2 and arule for 
finding a, from aj, a2,...,@,—1 forn = 3,4,5,.... 

b) Find the value of a, if aj) = 1, a2 =2, and a, = 
Qy—| + adn—-2 +--+: + a1, forn = 3,4,5,.... 
9. Give two examples of how well-formed formulae are de- 
fined recursively for different sets of elements and oper- 
ators. 


10. a) Give a recursive definition of the length of a string. 


b) Use the recursive definition from part (a) and struc- 
tural induction to prove that /(xy) = /(x) + /(y). 


11. a) What is a recursive algorithm? 
b) Describe a recursive algorithm for computing the sum 
of m numbers in a sequence. 


12. Describe a recursive algorithm for computing the greatest 
common divisor of two positive integers. 


13. a) Describe the merge sort algorithm. 
b) Use the merge sort algorithm to put the list 4, 10, 1, 5, 
3, 8, 7, 2, 6, 9 in increasing order. 
c) Give a big-O estimate for the number of comparisons 
used by the merge sort. 


14. a) Does testing a computer program to see whether it 
produces the correct output for certain input values 
verify that the program always produces the correct 
output? 

b) Does showing that a computer program is partially 
correct with respect to an initial assertion and a final 
assertion verify that the program always produces the 
correct output? If not, what else is needed? 


15. What techniques can you use to show that a long computer 
program is partially correct with respect to an initial as- 
sertion and a final assertion? 


16. What is a loop invariant? How is a loop invariant used? 


Supplementary Exercises 


1. 


12. 


13. 


14. 


15. 


16. 


17. 


cq 


Use mathematical induction to show that 5 + 5 + x + 


veep * =1- wn whenever 7 is a positive integer. 


Use mathematical induction to show that 13 + 33 + 53 + 
oe + (2n + 13 = (n+ 1)2(2n? + 4n + 1) whenever n 
is a positive integer. 


. Use mathematical induction to show that 1 - 2° + 2-2! + 


3-22 4..-4n-2"—-! = (n — 1)-2" + 1 whenever n is 
a positive integer. 
U 


se mathematical induction to show that 
1 1 1 n 


13° 3:5"  " Ga=DGie) mei 
whenever 7 is a positive integer. 
. Show that 
1 1 1 n 
—+ 75+ 


ST Gea Gnd tS 


whenever n is a positive integer. 


. Use mathematical induction to show that 2” > n2 +n 


whenever n is an integer greater than 4. 


. Use mathematical induction to show that 2” > n? when- 


ever n is an integer greater than 9. 


. Find an integer N such that 2” > n* whenever n is greater 


than N. Prove that your result is correct using mathemat- 
ical induction. 


. Use mathematical induction to prove that a — bis a factor 


° 


f a” — b” whenever n is a positive integer. 


e 


se mathematical induction to prove that 9 divides n? + 
n+1)3 + (n + 2)3 whenever n is a nonnegative integer. 


ant 


se mathematical induction to prove that 43 divides 
n+1 4 72n—1 for every positive integer n. 


CE ON 


se mathematical induction to prove that 64 divides 
2n+2 | 56n + 55 for every positive integer n. 


ioe) 


S 


se mathematical induction to prove this formula for the 
sum of the terms of an arithmetic progression. 


at(a+d)+---+(a+nd)= 
(n+ 1)Qa+nd)/2 


Suppose that a; = b; (mod m) for j = 1, 2,...,n. Use 
mathematical induction to prove that 


a) >» q= > bj (mod m). 


j=l j=l 
n n 
b) Il q= Il bj (mod m). 
j=l j=l 
Show that if n is a positive integer, then 
k+4 n(3n +7) 


Lk= EE Dk+2) 2n+Da+2)' 


For which positive integers n is n + 6 < (n* — 8n)/16? 
Prove your answer using mathematical induction. 

(Requires calculus) Suppose that f (x) = e* and g(x) = 
xe*. Use mathematical induction together with the prod- 


18. 


*19, 


*20. 


*21, 
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uct rule and the fact that f’(x) =e* to prove that 
g(x) = (x +n)e* whenever n is a positive integer. 


(Requires calculus) Suppose that f(x) = e* and g(x) = 
e“*, where c is aconstant. Use mathematical induction to- 
gether with the chain rule and the fact that f’(x) = e* to 
prove that g) = c”e°* whenever n is a positive integer. 


Formulate a conjecture about which Fibonacci numbers 
are even, and use a form of mathematical induction to 
prove your conjecture. 


Determine which Fibonacci numbers are divisible by 3. 
Use a form of mathematical induction to prove your con- 
jecture. 


Prove that fi fn + frei fntl = fn+k+1 for all nonnega- 
tive integers n and k, where f; denotes the ith Fibonacci 
number. 


Recall from Example 15 of Section 2.4 that the sequence 
of Lucas numbers is defined by Jp = 2, J) = 1, and J, = 
In—1 +In—2 forn = 2,3,4,.... 


22. 


23. 


*24, 


25. 


* 26. 


27. 


28. 


29. 


Show that fn + fn+2 = /n41 whenever n is a positive in- 
teger, where jf; and /; are the ith Fibonacci number and 
ith Lucas number, respectively. 


Show that /3 +17 + +++ +12 = Inln 41 +2 whenever n is 
a nonnegative integer and /; is the ith Lucas number. 


Use mathematical induction to show that the product of 
any n consecutive positive integers is divisible by n!. 
[Hint: Use the identity m(m+1)---(m+n— 
1)/n! = (m — 1)m(m + 1)--- (m+n — 2)/n!+ 
m(m+ 1)---(m+n—2)/(n— 1)!.] 

Use mathematical induction to show that (cosx + 
isinx)” = cosnx +isinnx whenever n is a positive in- 
teger. (Here i is the square root of —1.) [Hint: Use 
the identities cos(a + b) = cosacosb — sina sinb and 
sin(a + b) = sinacosb+cosasinb.] 

Use mathematical induction to show that ae cos jx = 
cos[(n + 1)x/2] sin(nx/2)/sin(x/2) whenever n is a 
positive integer and sin(x/2) 4 0. 

Use mathematical induction to prove that 2 4 Pas 
n22rtl _ n2nt+2 4 3.2"+1 _ 6 for every positive inte- 
gern. 


(Requires calculus) Suppose that the sequence 
X1,X2,...,%,... 18 recursively defined by x; = 0 and 


Xn+1 = Jxn +6. 
a) Use mathematical induction to show that xj < x2 < 
+ <Xp, <.---, thatis, the sequence {x,} is monoton- 

ically increasing. 

b) Use mathematical induction to prove that x, < 3 for 
n= 1,2,.... 

c) Show that limy_,.5 x, = 3. 

Show if 7 is a positive integer with n > 2, then 


3 1 @=)Gn+2) 
Pal~ 


=) 4n(n + 1). 
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30. 


*31. 


32. 


33. 


34. 


35. 


* 36. 


37. 


* 38. 


39. 


* 40. 


Use mathematical induction to prove Theorem 1 in Sec- 
tion 4.2, that is, show if b is an integer, where b > 1, and 
n is a positive integer, then n can be expressed uniquely 
in the form n = agb* + ag_1bk! + --» + a1b + ao. 

A lattice point in the plane is a point (x, y) where both x 
and y are integers. Use mathematical induction to show 
that at least n+ 1 straight lines are needed to ensure 
that every lattice point (x, y) with x > 0, y > 0, and 
x + y <n lies on one of these lines. 

(Requires calculus) Use mathematical induction and the 
product rule to show that if n is a positive integer and 
Fi(x), fox), ..., fr(x), are all differentiable functions, 
then 


(fix) fox) +++ fn) 
Six) fox) +++ fn) 
A@) AQ), @) 
fi) fa) f(x) 
(Requires material in Section 2.6) Suppose that B = 
MAM !, where A and B are n x n matrices and M is 
invertible. Show that Bf = MA‘M~! for all positive in- 


tegers k. (Consult both the text of Section 2.6 and the 
preamble to Exercise 18 of Section 2.6.) 


Use mathematical induction to show that if you draw lines 
in the plane you only need two colors to color the regions 
formed so that no two regions that have an edge in com- 
mon have a common color. 


Show that n! can be represented as the sum of n of its 
distinct positive divisors whenever n > 3. [Hint: Use in- 
ductive loading. First try to prove this result using mathe- 
matical induction. By examining where your proof fails, 
find a stronger statement that you can easily prove using 
mathematical induction. | 

Use mathematical induction to prove thatifx;, x2,... 
are positive real numbers with n > 2, then 


1 1 1 
xy + — Jl x2 + — Joe Xn + — |] Se 
x] x2 Xn 
1 1 1 1 
xy + — J x2 + — Je (Xn + — J an + — 
x2 x3 Xn x] 


Use mathematical induction to prove that ifn people stand 
in a line, where n is a positive integer, and if the first per- 
son in the line is a woman and the last person in line is a 
man, then somewhere in the line there is a woman directly 
in front of a man. 


Xn 


Suppose that for every pair of cities in a country there is a 
direct one-way road connecting them in one direction or 
the other. Use mathematical induction to show that there 
is a city that can be reached from every other city either 
directly or via exactly one other city. 

Use mathematical induction to show that when n circles 
divide the plane into regions, these regions can be col- 
ored with two different colors such that no regions with a 
common boundary are colored the same. 

Suppose that among a group of cars on a circular track 
there is enough fuel for one car to complete a lap. Use 
mathematical induction to show that there is a car in the 


41. 


42. 


* 43. 


group that can complete a lap by obtaining gas from other 
cars as it travels around the track. 


Show that if n is a positive integer, then 


n n 


YoQi-D| SO /k] Hat 1/2. 


j=l k=j 


Use mathematical induction to show that if a, b, and c 
are the lengths of the sides of a right triangle, where c is 
the length of the hypotenuse, then a” + b” < c" for all 
integers n with n > 3. 

Use mathematical induction to show that if is a posi- 

2 
tive integers, the sequence 2 mod n, 2* mod n, 2” mod n, 
22 F , 

2?" mod n,... is eventually constant (that is, all terms 
after a finite number of terms are all the same). 


44. A unit or Egyptian fraction is a fraction of the form 


1/n, where n is a positive integer. In this exercise, we 
will use strong induction to show that a greedy algorithm 
can be used to express every rational number p/q with 
0 < p/q < 1asthe sum of distinct unit fractions. At each 
step of the algorithm, we find the smallest positive integer 
n such that 1/n can be added to the sum without exceed- 
ing p/q. For example, to express 5/7 we first start the 
sum with 1/2. Because 5/7 — 1/2 = 3/14 we add 1/5 to 
the sum because 5 is the smallest positive integer k such 
that 1/k < 3/14. Because 3/14 — 1/5 = 1/70, the algo- 
rithm terminates, showing that 5/7 = 1/2 + 1/5 + 1/70. 
Let T(p) be the statement that this algorithm terminates 
for all rational numbers p/q with O < p/q < 1. We will 
prove that the algorithm always terminates by showing 
that T(p) holds for all positive integers p. 

a) Show that the basis step T (1) holds. 

b) Suppose that T(k) holds for positive integers k with 
k < p. That is, assume that the algorithm terminates 
for all rational numbers k/r, where 1 < k < p. Show 
that if we start with p/q and the fraction 1/n is se- 
lected in the first step of the algorithm, then p/q = 
p'/q' +1/n, where p’ = np — q and q’ = nq. After 
considering the case where p/q = 1/n, use the in- 
ductive hypothesis to show that the greedy algorithm 
terminates when it begins with p’/q’ and complete the 
inductive step. 


The McCarthy 91 function (defined by John McCarthy, one 
of the founders of artificial intelligence) is defined using the 


rule 


n—10 


ifn > 100 
se iene +11)) 


ifn < 100 


for all positive integers n. 


45. 


#46, 


By successively using the defining rule for M(n), find 
a) M(102). b) M(101). c) M(99). 

d) M(97). e) M(87). f) M(76). 

Show that the function M(n) is a well-defined function 
from the set of positive integers to the set of positive inte- 
gers. [Hint: Prove that M(n) = 91 for all positive integers 
nwithn < 101.] 


47. 


48. 


*49, 


*50. 


*51, 
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Is this proof that element a. Then show that a./2 — a is a smaller positive 
P 
1 1 1 3 1 integer of this form.] 

1-2 + 2.3 peter (n—1)n =H A? 52. A set is well ordered if every nonempty subset of this 
whenever n is a positive integer, correct? Justify your an- ae has e least element. Determine whether each of the 
ewer: following sets is well ordered. 

; : a) the set of integers 
Basis step: The result is true when n = | because b) the set of integers greater than — 100 
1321 c) the set of positive rationals 

12 2 21° d) the set of positive rationals with denominator less than 

Inductive step: Assume that the result is true for n. Then i 
1 1 1 1 53. a) Show that if aj,a2,...,d, are positive integers, 
a Se arc aa then gced(a1, a2, ..., Gn—1, An) = ged(ay, a2, ..., 
1-2 2-3 (1 — 1)n n(n + I) Qn—2, ECA(An—-1, An)). 
3» 1 1 1 b) Use part (a), together with the Euclidean algorithm, to 
~ ah n n+l develop arecursive algorithm for computing the great- 
3 1 est common divisor of a set of 1 positive integers. 
aq aq *54. Describe a recursive algorithm for writing the greatest 
: : : . . common divisor of 1 positive integers as a linear combi- 
— ee ae for n + 1 if it is true for n. This nation of these integers. 
i esa taal odie 55. Find an explicit formula for f(n) if f(1) = land f(n) = 
Suppose that Aj, Az,..., An are a collection of sets. f(n—1)+2n—1 for n> 2. Prove your result using 
ee that ie Ay : Ap bie ee: ® Ak “te mathematical induction. 
= 3,4, ...,.Use mathematical induction to prove tha ae ‘ : we by cages 
x € R, if and only if x belongs to an odd number of the 56. Give a recursive Sonne a the set of bit strings that 
sets Aj, A2,..., An. (Recall that S @ T is the symmetric CONE INAGS ae eel . a : 
difference of the sets S and T defined in the preamble to 57. Let S be the set of bit strings defined recursively by 4 € S 
Exercise 32 of Section 2.2.) and Ox € S,x1 € Sifx € S, where A is the empty string. 
Show that n circles divide the plane into n? — n + 2 re- a) Find all strings in S of length not exceeding five. 
gions if every two circles intersect in exactly two points b) Give an explicit description of the elements of S. 
and no three circles contain a common point. 58. Let S be the set of strings defined recursively by abc € S, 
Show that 1 planes divide three-dimensional space into bac € S,andacb € S, where a, b, and c are fixed letters; 
(n3 + 5n + 6)/6 regions if any three of these planes have and for all x € S, abex € S; abxc € S, axbe € S, and 
exactly one point in common and no four contain a com- xabc € S, where x is a variable representing a string of 
mon point. letters. 
Use the well-ordering property to show that J/2 is ir- a) Find all elements of S of length eight or less. 
rational. [Hint: Assume that /2 is rational. Show that b) Show that every element of S has a length divisible by 
the set of positive integers of the form b./2 has a least three. 


Links 


JOHN McCARTHY (BORN 1927) John McCarthy was born in Boston. He grew up in Boston and in Los 
Angeles. He studied mathematics as both an undergraduate and a graduate student, receiving his B.S. in 1948 from 
the California Institute of Technology and his Ph.D. in 1951 from Princeton. After graduating from Princeton, 
McCarthy held positions at Princeton, Stanford, Dartmouth, and M.LT. He held a position at Stanford from 1962 
until 1994, and is now an emeritus professor there. At Stanford, he was the director of the Artificial Intelligence 
Laboratory, held a named chair in the School of Engineering, and was a senior fellow in the Hoover Institution. 

McCarthy was a pioneer in the study of artificial intelligence, a term he coined in 1955. He worked 
| on problems related to the reasoning and information needs required for intelligent computer behavior. Mc- 
1 Carthy was among the first computer scientists to design time-sharing computer systems. He developed LISP, 
a programming language for computing using symbolic expressions. He played an important role in using logic to verify the correctness 
of computer programs. McCarthy has also worked on the social implications of computer technology. He is currently working on 
the problem of how people and computers make conjectures through assumptions that complications are absent from situations. 
McCarthy is an advocate of the sustainability of human progress and is an optimist about the future of humanity. He has also begun 
writing science fiction stories. Some of his recent writing explores the possibility that the world is a computer program written by 
some higher force. 

Among the awards McCarthy has won are the Turing Award from the Association for Computing Machinery, the Research 
Excellence Award of the International Conference on Artificial Intelligence, the Kyoto Prize, and the National Medal of Science. 
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The set B of all balanced strings of parentheses is defined 
recursively by 4 € B, where A is the empty string; (x) € B, 
xyeBifx,ye B. 


59. Show that (()()) is a balanced string of parentheses and 
(())) is not a balanced string of parentheses. 


60. Find all balanced strings of parentheses with exactly six 
symbols. 


61. Find all balanced strings of parentheses with four or fewer 
symbols. 


62. Use induction to show that if x is a balanced string of 
parentheses, then the number of left parentheses equals 
the number of right parentheses in x. 


Define the function N on the set of strings of parentheses by 


NA) =0,N() =1,NQ) =-1, 
N(uv) = N(u) + NQ), 


where A is the empty string, and u and v are strings. It can be 
shown that NV is well defined. 


63. Find 
a) N(()). b) NONOVM. 
ce) NCO). d) NOCO)M)). 
** 64, Show that a string w of parentheses is balanced if and only 


if N(@w) = 0 and N(u) => O whenever u is a prefix of w, 
that is, w = uv. 


*65. Give arecursive algorithm for finding all balanced strings 
of parentheses containing n or fewer symbols. 


66. Give a recursive algorithm for finding gcd(a, b), where 
a and b are nonnegative integers not both zero, 
based on these facts: gcd(a, b) = gcd(b, a) if a> b, 
gcd(0, b) = b, gcd(a, b) = 2 gcd(a/2, b/2) ifa and b are 
even, gcd(a, b) = gcd(a/2, b) if a is even and b is odd, 
and gcd(a, b) = gcd(a, b — a). 

67. Verify the program segment 

ifx > y then 
xi=y 
with respect to the initial assertion T and the final asser- 
tionx < y. 


*68. Develop a rule of inference for verifying recursive pro- 
grams and use it to verify the recursive algorithm for com- 
puting factorials given as Algorithm 1 in Section 5.4. 


Computer Projects 


Write programs with these input and output. 


** 1, Given a 2” x 2” checkerboard with one square missing, 
construct a tiling of this checkerboard using right triomi- 
noes. 


** 2. Generate all well-formed formulae for expressions in- 
volving the variables x, y, and z and the operators 
{+, *, /, —} with or fewer symbols. 


** 3, Generate all well-formed formulae for propositions with 
n or fewer symbols where each symbol is T, F, one of 


69. Devise a recursive algorithm that counts the number of 
times the integer 0 occurs in a list of integers. 
Exercises 70-77 deal with some unusual sequences, in- 
formally called self-generating sequences, produced by 
simple recurrence relations or rules. In particular, Exer- 
cises 70-75 deal with the sequence {a(n)} defined by 
a(n) =n—a(a(n —1)) for n> 1 and a(O) = O. (This se- 
quence, as well as those in Exercises 74 and 75, are defined 
in Douglas Hofstader’s fascinating book Gédel, Escher, Bach 
({Ho99}). 
70. Find the first 10 terms of the sequence {a(n)} defined in 
the preamble to this exercise. 


*71. Prove that this sequence is well defined. That is, show that 
a(n) is uniquely defined for all nonnegative integers n. 


** 72, Prove that a(n) = [(n + 1)u| where w = (—1 + 5) /2. 


[Hint: First show for all n > 0 that (un — |un]) + 
(u2n — |u2n|) = 1. Then show for all real numbers a 
withO <a <landa4A1—ypthat|(1+mu)d-—a@)J}+ 
la + «| = 1, considering the cases 0 < a < 1 — w and 
1—y <a <1 separately.] 

*73. Use the formula from Exercise 72 to show that 
a(n) =a(n—1) if un—|pun| <1—yp and a(n) = 
a(n — 1) + 1 otherwise. 

74. Find the first 10 terms of each of the following self- 
generating sequences: 
a) a(n) =n — a(a(a(n — 1))) forn = 1,a(0) = 0 
b) a(n) =n — a(a(a(a(n — 1)))) forn = 1, a(0) = 0 
ce) a(n) = a(n —a(n — 1)) + a(n — a(n — 2)) for n => 
3,a(1) = land a(2) = 1 
75. Find the first 10 terms of both the sequences m(n) and 
f() defined by the following pair of interwoven recur- 
rence relations: m(n) =n — f(m(n — 1)), f(r) =n — 
m(f(n — 1)) forn > 1, f(0) = 1 and m(0) = 0. 
Golomb’s self-generating sequence is the unique nonde- 
creasing sequence of positive integers a1, a2, a3, ... that has 
the property that it contains exactly a, occurrences of k for 
each positive integer k. 
76. Find the first 20 terms of Golomb’s self-generating se- 
quence. 

*77, Show that if f(m) is the largest integer m such 
that a, =n, where a, is the mth term of Golomb’s 
self-generating sequence, then f(n) =) “¢_1, ax and 


Ff @)) = Vas Kak. 


the propositional variables p and q, or an operator from 
[7 Vitae sel: 


4. Given a string, find its reversal. 


5. Given a real number a and a nonnegative integer n, find 
a” using recursion. 


6. Given a real number a and a nonnegative integer n, find 


n : : 
ae using recursion. 


#7. 


10. 


11. 


Given areal number a and a nonnegative integer n, find a” 
using the binary expansion of n and a recursive algorithm 


: k 
for computing a”. 


. Given two integers not both zero, find their greatest com- 


mon divisor using recursion. 


. Given a list of integers and an element x, locate x in this 


list using a recursive implementation of a linear search. 
Given a list of integers and an element x, locate x in this 
list using a recursive implementation of a binary search. 
Given a nonnegative integer n, find the nth Fibonacci 
number using iteration. 


Computations and Explorations 


12. 


13. 


14. 


15. 
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Given a nonnegative integer n, find the nth Fibonacci 
number using recursion. 


Given a positive integer, find the number of partitions of 
this integer. (See Exercise 47 of Section 5.3.) 


Given positive integers m and n, find A(m, n), the value of 
Ackermann’s function at the pair (m,n). (See the pream- 
ble to Exercise 48 of Section 5.3.) 


Given a list of n integers, sort these integers using the 
merge sort. 


Use a computational program or programs you have written to do these exercises. 


. What are the largest values of n for which n! has fewer than 


100 decimal digits and fewer than 1000 decimal digits? 


. Determine which Fibonacci numbers are divisible by 5, 


which are divisible by 7, and which are divisible by 11. 
Prove that your conjectures are correct. 


. Construct tilings using right triominoes of various 16 x 16, 


32 x 32,and 64 x 64 checkerboards with one square miss- 
ing. 


. Explore which m x n checkerboards can be completely 


Writing Projects 


Respond to these with essays using outside sources. 


. Describe the origins of mathematical induction. Who were 


the first people to use it and to which problems did they 
apply it? 


. Explain how to prove the Jordan curve theorem for sim- 


ple polygons and describe an algorithm for determining 
whether a point is in the interior or exterior of a simple 


polygon. 


. Describe how the triangulation of simple polygons is used 


in some key algorithms in computational geometry. 


ERS 


covered by right triominoes. Can you make a conjecture 
that answers this question? 


. Implement an algorithm for determining whether a point 


is in the interior or exterior of a simple polygon. 


. Implement an algorithm for triangulating a simple poly- 


gon. 


. Which values of Ackermann’s function are small enough 


that you are able to compute them? 


. Compare either the number of operations or the time 


needed to compute Fibonacci numbers recursively versus 
that needed to compute them iteratively. 


. Describe a variety of different applications of the Fibonacci 


numbers to the biological and the physical sciences. 


. Discuss the uses of Ackermann’s function both in the theory 


of recursive definitions and in the analysis of the complex- 
ity of algorithms for set unions. 


. Discuss some of the various methodologies used to es- 


tablish the correctness of programs and compare them to 
Hoare’s methods described in Section 5.5. 


. Explain how the ideas and concepts of program correctness 


can be extended to prove that operating systems are secure. 


6.1 


6.2 The Pigeonhole 


6.3 


6.4 


6.5 


6.6 


The Basics of 
Counting 


Principle 


Permutations 
and 
Combinations 


Binomial 
Coefficients 
and Identities 


Generalized 
Permutations 
and 
Combinations 


Generating 
Permutations 
and 
Combinations 


Counting 


C ombinatorics, the study of arrangements of objects, is an important part of discrete math- 
ematics. This subject was studied as long ago as the seventeenth century, when combi- 
natorial questions arose in the study of gambling games. Enumeration, the counting of objects 
with certain properties, is an important part of combinatorics. We must count objects to solve 
many different types of problems. For instance, counting is used to determine the complexity of 
algorithms. Counting is also required to determine whether there are enough telephone numbers 
or Internet protocol addresses to meet demand. Recently, it has played a key role in mathematical 
biology, especially in sequencing DNA. Furthermore, counting techniques are used extensively 
when probabilities of events are computed. 

The basic rules of counting, which we will study in Section 6.1, can solve a tremendous 
variety of problems. For instance, we can use these rules to enumerate the different telephone 
numbers possible in the United States, the allowable passwords on a computer system, and the 
different orders in which the runners in a race can finish. Another important combinatorial tool 
is the pigeonhole principle, which we will study in Section 6.2. This states that when objects are 
placed in boxes and there are more objects than boxes, then there is a box containing at least two 
objects. For instance, we can use this principle to show that among a set of 15 or more students, 
at least 3 were born on the same day of the week. 

We can phrase many counting problems in terms of ordered or unordered arrangements of 
the objects of a set with or without repetitions. These arrangements, called permutations and 
combinations, are used in many counting problems. For instance, suppose the 100 top finishers 
on a competitive exam taken by 2000 students are invited to a banquet. We can count the possible 
sets of 100 students that will be invited, as well as the ways in which the top 10 prizes can be 
awarded. 

Another problem in combinatorics involves generating all the arrangements of a specified 
kind. This is often important in computer simulations. We will devise algorithms to generate 
arrangements of various types. 


The Basics of Counting 


Introduction 


Suppose that a password on a computer system consists of six, seven, or eight characters. Each 
of these characters must be a digit or a letter of the alphabet. Each password must contain at least 
one digit. How many such passwords are there? The techniques needed to answer this question 
and a wide variety of other counting problems will be introduced in this section. 

Counting problems arise throughout mathematics and computer science. For example, we 
must count the successful outcomes of experiments and all the possible outcomes of these 
experiments to determine probabilities of discrete events. We need to count the number of 
operations used by an algorithm to study its time complexity. 

We will introduce the basic techniques of counting in this section. These methods serve as 
the foundation for almost all counting techniques. 
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Assessment 


Extra p> 
Examples 2 


EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


Basic Counting Principles 


We first present two basic counting principles, the product rule and the sum rule. Then we will 
show how they can be used to solve many different counting problems. 
The product rule applies when a procedure is made up of separate tasks. 


THE PRODUCT RULE Suppose that a procedure can be broken down into a sequence of 
two tasks. If there are n; ways to do the first task and for each of these ways of doing the first 
task, there are n2 ways to do the second task, then there are njn2 ways to do the procedure. 


Examples 1-10 show how the product rule is used. 


A new company with just two employees, Sanchez and Patel, rents a floor of a building with 
12 offices. How many ways are there to assign different offices to these two employees? 


Solution: The procedure of assigning offices to these two employees consists of assigning an 
office to Sanchez, which can be done in 12 ways, then assigning an office to Patel different from 
the office assigned to Sanchez, which can be done in |1 ways. By the product rule, there are 
12-11 = 132 ways to assign offices to these two employees. < 


The chairs of an auditorium are to be labeled with an uppercase English letter followed by a 
positive integer not exceeding 100. What is the largest number of chairs that can be labeled 
differently? 


Solution: The procedure of labeling a chair consists of two tasks, namely, assigning to the seat 
one of the 26 uppercase English letters, and then assigning to it one of the 100 possible integers. 
The product rule shows that there are 26 - 100 = 2600 different ways that a chair can be labeled. 
Therefore, the largest number of chairs that can be labeled differently is 2600. < 


There are 32 microcomputers in a computer center. Each microcomputer has 24 ports. How 
many different ports to a microcomputer in the center are there? 


Solution: The procedure of choosing a port consists of two tasks, first picking a microcomputer 
and then picking a port on this microcomputer. Because there are 32 ways to choose the micro- 
computer and 24 ways to choose the port no matter which microcomputer has been selected, 
the product rule shows that there are 32 - 24 = 768 ports. < 


An extended version of the product rule is often useful. Suppose that a procedure is carried 
out by performing the tasks 7), To, ..., Tj, in sequence. If each task T;,i = 1, 2,...,n, can be 
done in n; ways, regardless of how the previous tasks were done, then there aren; -n2- +--+ - Nm 
ways to carry out the procedure. This version of the product rule can be proved by mathematical 
induction from the product rule for two tasks (see Exercise 72). 


How many different bit strings of length seven are there? 
Solution: Each of the seven bits can be chosen in two ways, because each bit is either 0 or 1. 


Therefore, the product rule shows there are a total of 2”? = 128 different bit strings of length 
seven. < 


EXAMPLE 5 


26 ghoiaes 10 ghoices 
for each for each 
letter digit 


EXAMPLE 6 


EXAMPLE 7 


Counting the number of 
onto functions is harder. 
We’ll do this in Chapter 8. 


EXAMPLE 8 


Links 


Current projections are 
that by 2038, it will be 
necessary to add one or 
more digits to North 
American telephone 
numbers. 
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How many different license plates can be made if each plate contains a sequence of three 
uppercase English letters followed by three digits (and no sequences of letters are prohibited, 
even if they are obscene)? 


Solution: There are 26 choices for each of the three uppercase English letters and ten choices for 
each of the three digits. Hence, by the product rule there are a total of 26 - 26-26-10-10-10= 
17,576,000 possible license plates. < 


Counting Functions How many functions are there from a set with m elements to a set with 
n elements? 


Solution: A function corresponds to a choice of one of the n elements in the codomain for each of 
the m elements in the domain. Hence, by the product rule there aren-n- --- -n =n’ functions 
from a set with m elements to one with n elements. For example, there are 5° = 125 different 
functions from a set with three elements to a set with five elements. 


Counting One-to-One Functions How many one-to-one functions are there from a set with 
m elements to one with n elements? 


Solution: First note that when m > n there are no one-to-one functions from a set with m 
elements to a set with n elements. 

Now let m < n. Suppose the elements in the domain are a1, a2, ..., G. There are n ways 
to choose the value of the function at a,. Because the function is one-to-one, the value of the 
function at a2 can be picked inn — | ways (because the value used for a; cannot be used again). 
In general, the value of the function at a; can be chosen inn — k + | ways. By the product rule, 
there are n(n — 1)(n — 2)--- (n —m +1) one-to-one functions from a set with m elements to 
one with n elements. 

For example, there are 5 - 4- 3 = 60 one-to-one functions from a set with three elements to 
a set with five elements. 


The Telephone Numbering Plan The North American numbering plan (NANP) specifies the 
format of telephone numbers in the U.S., Canada, and many other parts of North America. A 
telephone number in this plan consists of 10 digits, which are split into a three-digit area code, a 
three-digit office code, and a four-digit station code. Because of signaling considerations, there 
are certain restrictions on some of these digits. To specify the allowable format, let X denote 
a digit that can take any of the values 0 through 9, let N denote a digit that can take any of 
the values 2 through 9, and let Y denote a digit that must be a 0 or a 1. Two numbering plans, 
which will be called the old plan, and the new plan, will be discussed. (The old plan, in use in 
the 1960s, has been replaced by the new plan, but the recent rapid growth in demand for new 
numbers for mobile phones and devices will eventually make even this new plan obsolete. In 
this example, the letters used to represent digits follow the conventions of the North American 
Numbering Plan.) As will be shown, the new plan allows the use of more numbers. 

In the old plan, the formats of the area code, office code, and station code are NYX, NNX, and 
XXXX, respectively, so that telephone numbers had the form NYX-NNX-XXXxX. In the new plan, 
the formats of these codes are NXX, NXX, and XXXX, respectively, so that telephone numbers 
have the form NXX-NXX-XXXX. How many different North American telephone numbers are 
possible under the old plan and under the new plan? 


Solution: By the product rule, there are 8-2-10 = 160 area codes with format NYX and 
8-10-10 = 800 area codes with format NXX. Similarly, by the product rule, there are 
8- 8-10 = 640 office codes with format NNX. The product rule also shows that there are 
10- 10-10-10 = 10,000 station codes with format XXXX. 
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Consequently, applying the product rule again, it follows that under the old plan there are 
160 - 640 - 10,000 = 1,024,000,000 

different numbers available in North America. Under the new plan, there are 
800 - 800 - 10,000 = 6,400,000,000 


different numbers available. < 


What is the value of k after the following code, where nj, 12, ..., Nm are positive integers, has 
been executed? 


k:=0 
for i; := | ton, 
for iz := 1 ton2 


for in := 1 ton 
k:=k+1 


Solution: The initial value of k is zero. Each time the nested loop is traversed, | is added 
to k. Let 7; be the task of traversing the ith loop. Then the number of times the loop is traversed 
is the number of ways to do the tasks 7, T>,..., Tj,. The number of ways to carry out the 
task T;, j = 1,2,...,m, is nj, because the jth loop is traversed once for each integer i; with 
1 <i; < nj. By the product rule, it follows that the nested loop is traversed njn2 - - - Nm times. 
Hence, the final value of k is njnz---nyp. < 


Counting Subsets of a Finite Set Use the product rule to show that the number of different 
subsets of a finite set S is 2!5!. 


Solution: Let S be a finite set. List the elements of S in arbitrary order. Recall from 
Section 2.2 that there is a one-to-one correspondence between subsets of S and bit strings 
of length ||. Namely, a subset of S is associated with the bit string with a | in the ith position if 
the ith element in the list is in the subset, and a 0 in this position otherwise. By the product rule, 
there are 2!5! bit strings of length |S]. Hence, | P(S)| = 2!5!. (Recall that we used mathematical 
induction to prove this fact in Example 10 of Section 5.1.) < 


The product rule is often phrased in terms of sets in this way: If A,, A2,..., Aj are finite 
sets, then the number of elements in the Cartesian product of these sets is the product of the 
number of elements in each set. To relate this to the product rule, note that the task of choosing 
an element in the Cartesian product A; x Az x --- X Am is done by choosing an element 
in Aj, an element in A2,..., and an element in A,,. By the product rule it follows that 


PD | | ll) | da 
DNA and Genomes _ The hereditary information of a living organism is encoded using de- 


oxyribonucleic acid (DNA), or in certain viruses, ribonucleic acid (RNA). DNA and RNA are 
extremely complex molecules, with different molecules interacting in a vast variety of ways to 
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enable living process. For our purposes, we give only the briefest description of how DNA and 
RNA encode genetic information. 

DNA molecules consist of two strands consisting of blocks known as nucleotides. Each 
nucleotide contains subcomponents called bases, each of which is adenine (A), cytosine (C), 
guanine (G), or thymine (T). The two strands of DNA are held together by hydrogen bonds 
connecting different bases, with A bonding only with T, and C bonding only with G. Unlike 
DNA, RNA is single stranded, with uracil (U) replacing thymine as a base. So, in DNA the 
possible base pairs are A-T and C-G, while in RNA they are A-U, and C-G. The DNA of a living 
creature consists of multiple pieces of DNA forming separate chromosomes. A gene is a segment 
of a DNA molecule that encodes a particular protein. The entirety of genetic information of an 
organism is called its genome. 

Sequences of bases in DNA and RNA encode long chains of proteins called amino acids. 
There are 22 essential amino acids for human beings. We can quickly see that a sequence of at 
least three bases are needed to encode these 22 different amino acid. First note, that because 
there are four possibilities for each base in DNA, A, C, G, and T, by the product rule there are 
4° = 16 < 22 different sequences of two bases. However, there are 4° = 64 different sequences 
of three bases, which provide enough different sequences to encode the 22 different amino acids 
(even after taking into account that several different sequences of three bases encode the same 
amino acid). 

The DNA of simple living creatures such as algae and bacteria have between 10° and 107 
links, where each link is one of the four possible bases. More complex organisms, such as in- 
sects, birds, and mammals have between 108 and 10!° links in their DNA. So, by the product 


iS ae s > : 
tule, there are at least 4! different sequences of bases in the DNA of simple organisms and at 


least 410° different sequences of bases in the DNA of more complex organisms. These are both 
incredibly huge numbers, which helps explain why there is such tremendous variability among 
living organisms. In the past several decades techniques have been developed for determining 
the genome of different organisms. The first step is to locate each gene in the DNA of an or- 
ganism. The next task, called gene sequencing, is the determination of the sequence of links 
on each gene. (Of course, the specific sequence of kinks on these genes depends on the partic- 
ular individual representative of a species whose DNA is analyzed.) For example, the human 
genome includes approximately 23,000 genes, each with 1,000 or more links. Gene sequencing 
techniques take advantage of many recently developed algorithms and are based on numerous 
new ideas in combinatorics. Many mathematicians and computer scientists work on problems 
involving genomes, taking part in the fast moving fields of bioinformatics and computational 
biology. < 


We now introduce the sum rule. 


THE SUM RULE Ifa task can be done either in one of mn, ways or in one of nz ways, where 
none of the set of n; ways is the same as any of the set of nz ways, then there are nj + n2 
ways to do the task. 


Example 12 illustrates how the sum rule is used. 


Suppose that either a member of the mathematics faculty or a student who is a mathematics major 
is chosen as a representative to a university committee. How many different choices are there 
for this representative if there are 37 members of the mathematics faculty and 83 mathematics 
majors and no one is both a faculty member and a student? 


Solution: There are 37 ways to choose a member of the mathematics faculty and there are 83 
ways to choose a student who is a mathematics major. Choosing a member of the mathematics 
faculty is never the same as choosing a student who is a mathematics major because no one is 
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both a faculty member and a student. By the sum rule it follows that there are 37 + 83 = 120 
possible ways to pick this representative. < 


We can extend the sum rule to more than two tasks. Suppose that a task can be done in one 
of n, ways, in one of nz ways, ... , or in one of n,, ways, where none of the set of n; ways of 
doing the task is the same as any of the set of n; ways, for all pairs i and j with 1 <i < j <m. 
Then the number of ways to do the task is nj +n2+----+nm. This extended version of the 
sum rule is often useful in counting problems, as Examples 13 and 14 show. This version of the 
sum rule can be proved using mathematical induction from the sum rule for two sets. (This is 
Exercise 71.) 


A student can choose a computer project from one of three lists. The three lists contain 23, 15, 
and 19 possible projects, respectively. No project is on more than one list. How many possible 
projects are there to choose from? 


Solution: The student can choose a project by selecting a project from the first list, the second 
list, or the third list. Because no project is on more than one list, by the sum rule there are 
23 + 15 + 19 = 57 ways to choose a project. < 


What is the value of k after the following code, where 11,72, ..., Mm are positive integers, has 
been executed? 


k:=0 

for i; := 1 ton, 
k:=k+1 

for i2 := 1 tono 
k:=k+1 


for i, := 1 ton 
k:=k+l1 


Solution: The initial value of k is zero. This block of code is made up of m different loops. 
Each time a loop is traversed, 1 is added to k. To determine the value of k after this code has 
been executed, we need to determine how many times we traverse a loop. Note that there are 
n; ways to traverse the ith loop. Because we only traverse one loop at a time, the sum rule 
shows that the final value of k, which is the number of ways to traverse one of the m loops is 
ny +tng+--- +N. 


The sum rule can be phrased in terms of sets as: If Aj, A2,..., Am are pairwise disjoint 
finite sets, then the number of elements in the union of these sets is the sum of the numbers of 
elements in the sets. To relate this to our statement of the sum rule, note there are | A;| ways to 
choose an element from A; for i = 1, 2,...,m. Because the sets are pairwise disjoint, when 
we select an element from one of the sets A;, we do not also select an element from a different 
set A;. Consequently, by the sum rule, because we cannot select an element from two of these 
sets at the same time, the number of ways to choose an element from one of the sets, which is 
the number of elements in the union, is 


|A1 U Ag U-+-UAm| = [Ai] + [Aol +--+ |Am| when A; 9 Aj = foralli, j. 


This equality applies only when the sets in question are pairwise disjoint. The situation is much 
more complicated when these sets have elements in common. That situation will be briefly 
discussed later in this section and discussed in more depth in Chapter 8. 
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More Complex Counting Problems 


Many counting problems cannot be solved using just the sum rule or just the product rule. 
However, many complicated counting problems can be solved using both of these rules in 
combination. We begin by counting the number of variable names in the programming language 
BASIC. (In the exercises, we consider the number of variable names in JAVA.) Then we will 
count the number of valid passwords subject to a particular set of restrictions. 


In a version of the computer language BASIC, the name of a variable is a string of one or 
two alphanumeric characters, where uppercase and lowercase letters are not distinguished. (An 
alphanumeric character is either one of the 26 English letters or one of the 10 digits.) Moreover, 
a variable name must begin with a letter and must be different from the five strings of two 
characters that are reserved for programming use. How many different variable names are there 
in this version of BASIC? 


Solution: Let V equal the number of different variable names in this version of BASIC. Let V; 
be the number of these that are one character long and V2 be the number of these that are two 
characters long. Then by the sumrule, V = V; + V2. Note that V; = 26, because a one-character 
variable name must be a letter. Furthermore, by the product rule there are 26 - 36 strings of length 
two that begin with a letter and end with an alphanumeric character. However, five of these 
are excluded, so V2 = 26-36 — 5 = 931. Hence, there are V = Vj + V2 = 26+ 931 = 957 
different names for variables in this version of BASIC. < 


Each user on a computer system has a password, which is six to eight characters long, where 
each character is an uppercase letter or a digit. Each password must contain at least one digit. 
How many possible passwords are there? 


Solution: Let P be the total number of possible passwords, and let P6, P7, and Pg denote 
the number of possible passwords of length 6, 7, and 8, respectively. By the sum rule, P = 
Po + P7 + Ps. We will now find Ps, P7, and Pg. Finding Pe¢ directly is difficult. To find Pe it is 
easier to find the number of strings of uppercase letters and digits that are six characters long, 
including those with no digits, and subtract from this the number of strings with no digits. By 
the product rule, the number of strings of six characters is 36°, and the number of strings with 
no digits is 26°. Hence, 


Po = 36° — 26° = 2,176,782,336 — 308,915,776 = 1,867,866, 560. 
Similarly, we have 

P, = 36’ — 26’ = 78,364, 164,096 — 8,031,810,176 = 70,332,353,920 
and 


Pg = 368 — 268 = 2,821,109,907,456 — 208,827,064,576 
= 2,612,282,842,880. 


Consequently, 
P= Po + P7+ Pg = 2,684,483,063,360. 4 


Counting Internet Addresses In the Internet, which is made up of interconnected physical 
networks of computers, each computer (or more precisely, each network connection of a com- 
puter) is assigned an Internet address. In Version 4 of the Internet Protocol (IPv4), now in use, 
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Bit Number 0 1 2 3 4 8 16 24 31 
Class A 0 netid hostid 

Class B 1 0 netid hostid 

Class C 1 1 0 netid hostid 

Class D 1 1 1 0 Multicast Address 

Class E 1 1 1 1 0 Address 


FIGURE 1 Internet Addresses (IPv4). 


an address is a string of 32 bits. It begins with a network number (netid ). The netid is followed 
by a host number (hostid ), which identifies a computer as a member of a particular network. 

Three forms of addresses are used, with different numbers of bits used for netids and hostids. 
Class A addresses, used for the largest networks, consist of 0, followed by a 7-bit netid and a 
24-bit hostid. Class B addresses, used for medium-sized networks, consist of 10, followed by 
a 14-bit netid and a 16-bit hostid. Class C addresses, used for the smallest networks, consist 
of 110, followed by a 21-bit netid and an 8-bit hostid. There are several restrictions on addresses 
because of special uses: 1111111 is not available as the netid of a Class A network, and the 
hostids consisting of all Os and all Is are not available for use in any network. A computer 
on the Internet has either a Class A, a Class B, or a Class C address. (Besides Class A, B, 
and C addresses, there are also Class D addresses, reserved for use in multicasting when mul- 
tiple computers are addressed at a single time, consisting of 1110 followed by 28 bits, and 
Class E addresses, reserved for future use, consisting of 11110 followed by 27 bits. Neither 
Class D nor Class E addresses are assigned as the IPv4 address of a computer on the Internet.) 
Figure | illustrates [Pv4 addressing. (Limitations on the number of Class A and Class B netids 
have made IPv4 addressing inadequate; IPv6, a new version of IP, uses 128-bit addresses to 
solve this problem.) 

How many different IPv4 addresses are available for computers on the Internet? 


Solution: Let x be the number of available addresses for computers on the Internet, and let x4, 
xg, and xc denote the number of Class A, Class B, and Class C addresses available, respectively. 
By the sum rule, x = x4 +xg4+Xc. 

To find x4, note that there are 2’ — 1 = 127 Class A netids, recalling that the netid 
1111111 is unavailable. For each netid, there are 274 — 2 = 16,777,214 hostids, recalling that the 
hostids consisting of all Os and all 1s are unavailable. Consequently, x4 = 127 - 16,777,214 = 
2,130,706, 178. 

To find xg and xc, note that there are 2!4 — 16,384 Class B netids and 22! = 2,097,152 
Class C netids. For each Class B netid, there are 2!© —2 = 65,534 hostids, and for each 
Class C netid, there are 28 _ 2 = 254 hostids, recalling that in each network the hostids 
consisting of all Os and all 1s are unavailable. Consequently, xg = 1,073,709,056 and xc = 
532,676,608. 

We conclude that the total number of IPv4 addresses available is x = x4 + xg + xc = 
2,130,706, 178 + 1,073,709,056 + 532,676,608 = 3,737,091,842. 


The Subtraction Rule (Inclusion—Exclusion for Two Sets) 


Suppose that a task can be done in one of two ways, but some of the ways to do it are common 
to both ways. In this situation, we cannot use the sum rule to count the number of ways to do 
the task. If we add the number of ways to do the tasks in these two ways, we get an overcount 
of the total number of ways to do it, because the ways to do the task that are common to the two 
ways are counted twice. To correctly count the number of ways to do the two tasks, we must 
subtract the number of ways that are counted twice. This leads us to an important counting rule. 
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THE SUBTRACTION RULE Ifa task can be done in either n; ways or nz ways, then the 
number of ways to do the task is nj +2 minus the number of ways to do the task that are 
common to the two different ways. 


The subtraction rule is also known as the principle of inclusion—exclusion, especially when 
it is used to count the number of elements in the union of two sets. Suppose that A; and A2 are 
sets. Then, there are |A;| ways to select an element from A, and |A2| ways to select an element 
from A2. The number of ways to select an element from A; or from Ao, that is, the number of 
ways to select an element from their union, is the sum of the number of ways to select an element 
from A, and the number of ways to select an element from Az, minus the number of ways to 
select an element that is in both A; and Az. Because there are |A; U A2| ways to select an element 
in either A; or in Az, and |A; M A2| ways to select an element common to both sets, we have 


[Ag 49) = 149) + |Ao| = [aq 19). 


This is the formula given in Section 2.2 for the number of elements in the union of two sets. 
Example 18 illustrates how we can solve counting problems using the subtraction principle. 


How many bit strings of length eight either start with a | bit or end with the two bits 00? 


Solution: We can construct a bit string of length eight that either starts with a | bit or ends 
with the two bits 00, by constructing a bit string of length eight beginning with a | bit or by 
constructing a bit string of length eight that ends with the two bits 00. We can construct a bit 
string of length eight that begins with a 1 in 2’ = 128 ways. This follows by the product rule, 
because the first bit can be chosen in only one way and each of the other seven bits can be 
chosen in two ways. Similarly, we can construct a bit string of length eight ending with the two 
bits 00, in 2° = 64 ways. This follows by the product rule, because each of the first six bits can 
be chosen in two ways and the last two bits can be chosen in only one way. 

Some of the ways to construct a bit string of length eight starting with a | are the same 
as the ways to construct a bit string of length eight that ends with the two bits 00. There are 
2° = 32 ways to construct such a string. This follows by the product rule, because the first 
bit can be chosen in only one way, each of the second through the sixth bits can be chosen 
in two ways, and the last two bits can be chosen in one way. Consequently, the number of 
bit strings of length eight that begin with a 1 or end with a 00, which equals the number 
of ways to construct a bit string of length eight that begins with a | or that ends with 00, 
equals 128 + 64 — 32 = 160. < 


We present an example that illustrates how the formulation of the principle of inclusion— 
exclusion can be used to solve counting problems. 


A computer company receives 350 applications from computer graduates for a job planning a 
line of new Web servers. Suppose that 220 of these applicants majored in computer science, 147 
majored in business, and 51 majored both in computer science and in business. How many of 
these applicants majored neither in computer science nor in business? 


Solution: To find the number of these applicants who majored neither in computer science nor 
in business, we can subtract the number of students who majored either in computer science 
or in business (or both) from the total number of applicants. Let A; be the set of students who 
majored in computer science and A2 the set of students who majored in business. Then A; U A2 
is the set of students who majored in computer science or business (or both), and A; M A2 is the 
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set of students who majored both in computer science and in business. By the subtraction rule 
the number of students who majored either in computer science or in business (or both) equals 


|Ay U Ag| = [Aq] + |A2| — |A1 9 Ag| = 220 + 147 — 51 = 316. 


We conclude that 350 — 316 = 34 of the applicants majored neither in computer science nor in 
business. < 


The subtraction rule, or the principle of inclusion—exclusion, can be generalized to find the 
number of ways to do one of n different tasks or, equivalently, to find the number of elements 
in the union of n sets, whenever n is a positive integer. We will study the inclusion—exclusion 
principle and some of its many applications in Chapter 8. 


The Division Rule 


We have introduced the product, sum, and subtraction rules for counting. You may wonder 
whether there is also a division rule for counting. In fact, there is such a rule, which can be 
useful when solving certain types of enumeration problems. 


THE DIVISION RULE There are n/d ways to do a task if it can be done using a procedure 
that can be carried out in n ways, and for every way w, exactly d of the n ways correspond 
to way w. 


We can restate the division rule in terms of sets: “If the finite set A is the union of n pairwise 
disjoint subsets each with d elements, then n = |A|/d.” 

We can also formulate the division rule in terms of functions: “If f is a function from A 
to B where A and B are finite sets, and that for every value y € B there are exactly d values 
x € A such that f(x) = y (in which case, we say that f is d-to-one), then |B| = |A|/d.” 

We illustrate the use of the division rule for counting with an example. 


How many different ways are there to seat four people around a circular table, where two 
seatings are considered the same when each person has the same left neighbor and the same 
right neighbor? 


Solution: We arbitrarily select a seat at the table and label it seat 1. We number the rest of the 
seats in numerical order, proceeding clockwise around the table. Note that are four ways to 
select the person for seat 1, three ways to select the person for seat 2, two ways to select the 
person for seat 3, and one way to select the person for seat 4. Thus, there are 4! = 24 ways to 
order the given four people for these seats. However, each of the four choices for seat | leads 
to the same arrangement, as we distinguish two arrangements only when one of the people has 
a different immediate left or immediate right neighbor. Because there are four ways to choose 
the person for seat 1, by the division rule there are 24/4 = 6 different seating arrangements of 
four people around the circular table. 


Tree Diagrams 


Counting problems can be solved using tree diagrams. A tree consists of a root, a number 
of branches leaving the root, and possible additional branches leaving the endpoints of other 
branches. (We will study trees in detail in Chapter 11.) To use trees in counting, we use a branch 
to represent each possible choice. We represent the possible outcomes by the leaves, which are 
the endpoints of branches not having other branches starting at them. 

Note that when a tree diagram is used to solve a counting problem, the number of choices 
of which branch to follow to reach a leaf can vary (see Example 21, for example). 
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FIGURE3 _ Best Three Games Out of Five Playoffs. 
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EXAMPLE 23 


How many bit strings of length four do not have two consecutive |s? 


Solution: The tree diagram in Figure 2 displays all bit strings of length four without two con- 
secutive 1s. We see that there are eight bit strings of length four without two consecutive ls. 4 


A playoff between two teams consists of at most five games. The first team that wins three games 
wins the playoff. In how many different ways can the playoff occur? 


Solution: The tree diagram in Figure 3 displays all the ways the playoff can proceed, with the 
winner of each game shown. We see that there are 20 different ways for the playoff to occur. 4 


Suppose that “I Love New Jersey” T-shirts come in five different sizes: S, M, L, XL, and XXL. 
Further suppose that each size comes in four colors, white, red, green, and black, except for XL, 
which comes only in red, green, and black, and XXL, which comes only in green and black. How 
many different shirts does a souvenir shop have to stock to have at least one of each available 


size and color of the T-shirt? 


Solution: The tree diagram in Figure 4 displays all possible size and color pairs. It follows that 
the souvenir shop owner needs to stock 17 different T-shirts. < 


W = white, R = red, G = green, B = black 


iL XL XXL 


WR: G BW RG BWR Oo BR. G BG B 


FIGURE 4 Counting Varieties of T-Shirts. 
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Exercises 


. There are 18 mathematics majors and 325 computer sci- 


ence majors at a college. 

a) In how many ways can two representatives be picked 
so that one is a mathematics major and the other is a 
computer science major? 


b) In how many ways can one representative be picked 
who is either a mathematics major or a computer sci- 
ence major? 


. An office building contains 27 floors and has 37 offices 


on each floor. How many offices are in the building? 


. A multiple-choice test contains 10 questions. There are 


four possible answers for each question. 


a) In how many ways can a student answer the questions 
on the test if the student answers every question? 


b) In how many ways can a student answer the questions 
on the test if the student can leave answers blank? 


. A particular brand of shirt comes in 12 colors, has a male 


version and a female version, and comes in three sizes 
for each sex. How many different types of this shirt are 
made? 


. Six different airlines fly from New York to Denver and 


seven fly from Denver to San Francisco. How many dif- 
ferent pairs of airlines can you choose on which to book 
a trip from New York to San Francisco via Denver, when 
you pick an airline for the flight to Denver and an airline 
for the continuation flight to San Francisco? 


. There are four major auto routes from Boston to Detroit 


and six from Detroit to Los Angeles. How many major 
auto routes are there from Boston to Los Angeles via De- 
troit? 


7. How many different three-letter initials can people have? 


13. 


14. 


15. 


16. 


. How many different three-letter initials with none of the 


letters repeated can people have? 


. How many different three-letter initials are there that be- 


gin with an A? 


. How many bit strings are there of length eight? 


. How many bit strings of length ten both begin and end 


with a 1? 


. How many bit strings are there of length six or less, not 


counting the empty string? 


How many bit strings with length not exceeding n, where 
n is a positive integer, consist entirely of 1s, not counting 
the empty string? 


How many bit strings of length n, where n is a positive 
integer, start and end with 1s? 


How many strings are there of lowercase letters of length 
four or less, not counting the empty string? 


How many strings are there of four lowercase letters that 
have the letter x in them? 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


How many strings of five ASCII characters contain the 

character @ (“at” sign) at least once? [Note: There are 

128 different ASCII characters. 

How many 5-element DNA sequences 

a) end with A? 

b) start with T and end with G? 

c) contain only A and T? 

d) do not contain C? 

How many 6-element RNA sequences 

a) do not contain U? 

b) end with GU? 

c) start with C? 

d) contain only A or U? 

How many positive integers between 5 and 31 

a) are divisible by 3? Which integers are these? 

b) are divisible by 4? Which integers are these? 

c) are divisible by 3 and by 4? Which integers are these? 

How many positive integers between 50 and 100 

a) are divisible by 7? Which integers are these? 

b) are divisible by 11? Which integers are these? 

c) are divisible by both 7 and 11? Which integers are 
these? 

How many positive integers less than 1000 

a) are divisible by 7? 

b) are divisible by 7 but not by 11? 

c) are divisible by both 7 and 11? 

d) are divisible by either 7 or 11? 

e) are divisible by exactly one of 7 and 11? 

f) are divisible by neither 7 nor 11? 

g) have distinct digits? 

h) have distinct digits and are even? 

How many positive integers between 100 and 999 inclu- 

sive 

a) are divisible by 7? 

b) are odd? 

c) have the same three decimal digits? 

d) are not divisible by 4? 

e) are divisible by 3 or 4? 

f) are not divisible by either 3 or 4? 

g) are divisible by 3 but not by 4? 

h) are divisible by 3 and 4? 

How many positive integers between 1000 and 9999 in- 

clusive 

a) are divisible by 9? 

b) are even? 

c) have distinct digits? 

d) are not divisible by 3? 

e) are divisible by 5 or 7? 

f) are not divisible by either 5 or 7? 

g) are divisible by 5 but not by 7? 

h) are divisible by 5 and 7? 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


How many strings of three decimal digits 

a) do not contain the same digit three times? 
b) begin with an odd digit? 

c) have exactly two digits that are 4s? 

How many strings of four decimal digits 


a) do not contain the same digit twice? 

b) end with an even digit? 

c) have exactly three digits that are 9s? 

A committee is formed consisting of one representative 
from each of the 50 states in the United States, where the 
representative from a state is either the governor or one 
of the two senators from that state. How many ways are 
there to form this committee? 


How many license plates can be made using either three 
digits followed by three uppercase English letters or three 
uppercase English letters followed by three digits? 


How many license plates can be made using either two 

uppercase English letters followed by four digits or two 

digits followed by four uppercase English letters? 

How many license plates can be made using either three 

uppercase English letters followed by three digits or four 

uppercase English letters followed by two digits? 

How many license plates can be made using either two 

or three uppercase English letters followed by either two 

or three digits? 

How many strings of eight uppercase English letters are 

there 

a) if letters can be repeated? 

b) if no letter can be repeated? 

c) that start with X, if letters can be repeated? 

d) that start with X, if no letter can be repeated? 

e) that start and end with X, if letters can be repeated? 

f) that start with the letters BO (in that order), if letters 
can be repeated? 

g) that start and end with the letters BO (in that order), 
if letters can be repeated? 

h) that start or end with the letters BO (in that order), if 
letters can be repeated? 

How many strings of eight English letters are there 

a) that contain no vowels, if letters can be repeated? 

b) that contain no vowels, if letters cannot be repeated? 

c) that start with a vowel, if letters can be repeated? 

d) that start with a vowel, if letters cannot be repeated? 

e) that contain at least one vowel, if letters can be re- 
peated? 

f) that contain exactly one vowel, if letters can be re- 
peated? 

g) that start with X and contain at least one vowel, if 
letters can be repeated? 

h) that start and end with X and contain at least one 
vowel, if letters can be repeated? 

How many different functions are there from a set with 

10 elements to sets with the following numbers of ele- 

ments? 

a) 2 b) 3 


c) 4 d) 5 


35. 


36. 


37. 


38. 


39. 


40. 


41. 


42. 


43. 


44. 


45. 


46. 


47. 
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How many one-to-one functions are there from a set with 
five elements to sets with the following number of ele- 
ments? 


a) 4 b) 5 c) 6 d) 7 

How many functions are there from the set {1, 2,..., n}, 
where n is a positive integer, to the set {0, 1}? 

How many functions are there from the set {1, 2, ..., n}, 


where n is a positive integer, to the set {0, 1} 

a) that are one-to-one? 

b) that assign 0 to both | and n? 

c) that assign 1 to exactly one of the positive integers 
less than n? 


How many partial functions (see Section 2.3) are there 
from a set with five elements to sets with each of these 
number of elements? 

a) l b) 2 c) 5 d) 9 

How many partial functions (see Definition 13 of Section 
2.3) are there from a set with m elements to a set with n 
elements, where m and n are positive integers? 


How many subsets of a set with 100 elements have more 
than one element? 


A palindrome is a string whose reversal is identical to the 
string. How many bit strings of length n are palindromes? 


How many 4-element DNA sequences 


a) do not contain the base T? 

b) contain the sequence ACG? 

c) contain all four bases A, T, C, and G? 

d) contain exactly three of the four bases A, T, C, and G? 


How many 4-element RNA sequences 


a) contain the base U? 

b) do not contain the sequence CUG? 

c) do not contain all four bases A, U, C, and G? 

d) contain exactly two of the four bases A, U, C, and G? 


How many ways are there to seat four of a group of ten 
people around a circular table where two seatings are con- 
sidered the same when everyone has the same immediate 
left and immediate right neighbor? 


How many ways are there to seat six people around a cir- 
cular table where two seatings are considered the same 
when everyone has the same two neighbors without re- 
gard to whether they are right or left neighbors? 


In how many ways can a photographer at a wedding ar- 
range 6 people in a row from a group of 10 people, where 
the bride and the groom are among these 10 people, if 


a) the bride must be in the picture? 

b) both the bride and groom must be in the picture? 

c) exactly one of the bride and the groom is in the pic- 
ture? 


In how many ways can a photographer at a wedding ar- 

range six people in a row, including the bride and groom, 

if 

a) the bride must be next to the groom? 

b) the bride is not next to the groom? 

c) the bride is positioned somewhere to the left of the 
groom? 
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48. 


49. 


*50. 


** 51, 


52. 


53. 


54. 


55. 


56. 


57. 


58. 


How many bit strings of length seven either begin with 
two Os or end with three 1s? 


How many bit strings of length 10 either begin with three 
Os or end with two 0s? 


How many bit strings of length 10 contain either five con- 
secutive Os or five consecutive 1s? 


How many bit strings of length eight contain either three 
consecutive 0s or four consecutive 1s? 


Every student in a discrete mathematics class is either a 
computer science or a mathematics major or is a joint 
major in these two subjects. How many students are in 
the class if there are 38 computer science majors (includ- 
ing joint majors), 23 mathematics majors (including joint 
majors), and 7 joint majors? 


How many positive integers not exceeding 100 are divis- 
ible either by 4 or by 6? 


How many different initials can someone have if a person 
has at least two, but no more than five, different initials? 
Assume that each initial is one of the 26 uppercase letters 
of the English language. 


Suppose that a password for a computer system must have 

at least 8, but no more than 12, characters, where each 

character in the password is a lowercase English letter, 

an uppercase English letter, a digit, or one of the six spe- 

cial characters *, >, <, !, +,and=. 

a) How many different passwords are available for this 
computer system? 


b) How many of these passwords contain at least one oc- 
currence of at least one of the six special characters? 


c) Using your answer to part (a), determine how long it 
takes a hacker to try every possible password, assum- 
ing that it takes one nanosecond for a hacker to check 
each possible password. 


The name of a variable in the C programming language is 
a string that can contain uppercase letters, lowercase let- 
ters, digits, or underscores. Further, the first character in 
the string must be a letter, either uppercase or lowercase, 
or an underscore. If the name of a variable is determined 
by its first eight characters, how many different variables 
can be named in C? (Note that the name of a variable may 
contain fewer than eight characters.) 


The name of a variable in the JAVA programming lan- 
guage is a string of between | and 65,535 characters, 
inclusive, where each character can be an uppercase or a 
lowercase letter, a dollar sign, an underscore, or a digit, 
except that the first character must not be a digit. Deter- 
mine the number of different variable names in JAVA. 


The International Telecommunications Union (ITU) 
specifies that a telephone number must consist of a coun- 
try code with between | and 3 digits, except that the code 0 
is not available for use as a country code, followed by 
a number with at most 15 digits. How many available 
possible telephone numbers are there that satisfy these 
restrictions? 


59. 


60. 


61. 


62. 


63. 


64. 


65. 


66. 


67. 


68. 


69. 


*70. 


Suppose that at some future time every telephone in the 
world is assigned a number that contains a country code 
1 to 3 digits long, that is, of the form X, XX, or XXX, 
followed by a 10-digit telephone number of the form 
NXX-NXX-XXXX (as described in Example 8). How 
many different telephone numbers would be available 
worldwide under this numbering plan? 

A key in the Vigenére cryptosystem is a string of English 
letters, where the case of the letters does not matter. How 
many different keys for this cryptosystem are there with 
three, four, five, or six letters? 


A wired equivalent privacy (WEP) key for a wireless fi- 
delity (WiFi) network is a string of either 10, 26, or 58 
hexadecimal digits. How many different WEP keys are 
there? 


Suppose that p and g are prime numbers and thatn = pq. 
Use the principle of inclusion—exclusion to find the num- 
ber of positive integers not exceeding n that are relatively 
prime to n. 


Use the principle of inclusion—exclusion to find the num- 
ber of positive integers less than 1,000,000 that are not 
divisible by either 4 or by 6. 


Use a tree diagram to find the number of bit strings of 
length four with no three consecutive Os. 


How many ways are there to arrange the letters a, b, c, 
and d such that a is not followed immediately by b? 


Use a tree diagram to find the number of ways that the 
World Series can occur, where the first team that wins 
four games out of seven wins the series. 


Use a tree diagram to determine the number of subsets 
of {3, 7,9, 11, 24} with the property that the sum of the 
elements in the subset is less than 28. 


a) Suppose that a store sells six varieties of soft drinks: 
cola, ginger ale, orange, root beer, lemonade, and 
cream soda. Use a tree diagram to determine the num- 
ber of different types of bottles the store must stock to 
have all varieties available in all size bottles if all vari- 
eties are available in 12-ounce bottles, all but lemon- 
ade are available in 20-ounce bottles, only cola and 
ginger ale are available in 32-ounce bottles, and all but 
lemonade and cream soda are available in 64-ounce 
bottles? 

b) Answer the question in part (a) using counting rules. 


a) Suppose that a popular style of running shoe is avail- 
able for both men and women. The woman’s shoe 
comes in sizes 6, 7, 8, and 9, and the man’s shoe 
comes in sizes 8, 9, 10, 11, and 12. The man’s shoe 
comes in white and black, while the woman’s shoe 
comes in white, red, and black. Use a tree diagram to 
determine the number of different shoes that a store 
has to stock to have at least one pair of this type of 
running shoe for all available sizes and colors for both 
men and women. 

b) Answer the question in part (a) using counting rules. 


Use the product rule to show that there are 22" different 
truth tables for propositions in n variables. 


71. 


72. 


73. 


74, 


Use mathematical induction to prove the sum rule for m 
tasks from the sum rule for two tasks. 

Use mathematical induction to prove the product rule 
for m tasks from the product rule for two tasks. 

How many diagonals does a convex polygon with n sides 
have? (Recall that a polygon is convex if every line seg- 
ment connecting two points in the interior or boundary of 
the polygon lies entirely within this set and that a diago- 
nal of a polygon is a line segment connecting two vertices 
that are not adjacent.) 

Data are transmitted over the Internet in datagrams, 
which are structured blocks of bits. Each datagram con- 
tains header information organized into a maximum 
of 14 different fields (specifying many things, including 
the source and destination addresses) and a data area that 
contains the actual data that are transmitted. One of the 
14 header fields is the header length field (denoted by 
HLEN), which is specified by the protocol to be 4 bits 
long and that specifies the header length in terms of 32-bit 
blocks of bits. For example, if HLEN = 0110, the header 
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is made up of six 32-bit blocks. Another of the 14 header 

fields is the 16-bit-long total length field (denoted by 

TOTAL LENGTH), which specifies the length in bits 

of the entire datagram, including both the header fields 

and the data area. The length of the data area is the total 
length of the datagram minus the length of the header. 

a) The largest possible value of TOTAL LENGTH 
(which is 16 bits long) determines the maximum 
total length in octets (blocks of 8 bits) of an Internet 
datagram. What is this value? 

b) The largest possible value of HLEN (which is 4 bits 
long) determines the maximum total header length 
in 32-bit blocks. What is this value? What is the 
maximum total header length in octets? 

c) The minimum (and most common) header length is 
20 octets. What is the maximum total length in octets 
of the data area of an Internet datagram? 

d) How many different strings of octets in the data area 
can be transmitted if the header length is 20 octets 
and the total length is as long as possible? 


Introduction 


Links 


Suppose that a flock of 20 pigeons flies into a set of 19 pigeonholes to roost. Because there are 
20 pigeons but only 19 pigeonholes, a least one of these 19 pigeonholes must have at least two 


pigeons in it. To see why this is true, note that if each pigeonhole had at most one pigeon in it, 
at most 19 pigeons, one per hole, could be accommodated. This illustrates a general principle 
called the pigeonhole principle, which states that if there are more pigeons than pigeonholes, 
then there must be at least one pigeonhole with at least two pigeons in it (see Figure 1). Of 
course, this principle applies to other objects besides pigeons and pigeonholes. 


THEOREM 1 


THE PIGEONHOLE PRINCIPLE If k is a positive integer and k + 1 or more objects 


are placed into k boxes, then there is at least one box containing two or more of the objects. 


O\E\E |e 
EE |e |e 


EE \E |e 


(a) (b) 


FIGURE 1 There Are More Pigeons Than Pigeonholes. 


(c) 
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COROLLARY 1 


EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


Proof: We prove the pigeonhole principle using a proof by contraposition. Suppose that none of 
the k boxes contains more than one object. Then the total number of objects would be at most k. 
This is a contradiction, because there are at least k + | objects. | 


The pigeonhole principle is also called the Dirichlet drawer principle, after the nineteenth- 
century German mathematician G. Lejeune Dirichlet, who often used this principle in his work. 
(Dirichlet was not the first person to use this principle; a demonstration that there were at least 
two Parisians with the same number of hairs on their heads dates back to the 17th century— 
see Exercise 33.) It is an important additional proof technique supplementing those we have 
developed in earlier chapters. We introduce it in this chapter because of its many important 
applications to combinatorics. 

We will illustrate the usefulness of the pigeonhole principle. We first show that it can be 
used to prove a useful corollary about functions. 


A function f froma set with k + 1 or more elements to a set with k elements is not one-to-one. 


Proof: Suppose that for each element y in the codomain of f we have a box that contains all 
elements x of the domain of f such that f(x) = y. Because the domain contains k + | or more 
elements and the codomain contains only k elements, the pigeonhole principle tells us that one 
of these boxes contains two or more elements x of the domain. This means that f cannot be 
one-to-one. dq 


Examples 1-3 show how the pigeonhole principle is used. 


Among any group of 367 people, there must be at least two with the same birthday, because 
there are only 366 possible birthdays. 


In any group of 27 English words, there must be at least two that begin with the same letter, 
because there are 26 letters in the English alphabet. < 


How many students must be in a class to guarantee that at least two students receive the same 
score on the final exam, if the exam is graded on a scale from 0 to 100 points? 


Solution: There are 101 possible scores on the final. The pigeonhole principle shows that among 
any 102 students there must be at least 2 students with the same score. < 


G. LEJEUNE DIRICHLET (1805-1859) G. Lejeune Dirichlet was born into a Belgian family living near 
Cologne, Germany. His father was a postmaster. He became passionate about mathematics at a young age. He 
was spending all his spare money on mathematics books by the time he entered secondary school in Bonn at the 
age of 12. At 14 he entered the Jesuit College in Cologne, and at 16 he began his studies at the University of Paris. 
In 1825 he returned to Germany and was appointed to a position at the University of Breslau. In 1828 he moved 
to the University of Berlin. In 1855 he was chosen to succeed Gauss at the University of Gottingen. Dirichlet 
is said to be the first person to master Gauss’s Disquisitiones Arithmeticae, which appeared 20 years earlier. 
He is said to have kept a copy at his side even when he traveled. Dirichlet made many important discoveries 


in number theory, including the theorem that there are infinitely many primes in arithmetical progressions an + b when a and b are 


relatively prime. He proved the n = 5 case of Fermat’s last theorem, that there are no nontrivial solutions in integers to x4 y> =2 


> 


Dirichlet also made many contributions to analysis. Dirichlet was considered to be an excellent teacher who could explain ideas with 
great clarity. He was married to Rebecca Mendelssohn, one of the sisters of the composer Frederick Mendelssohn. 


EXAMPLE 4 
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The pigeonhole principle is a useful tool in many proofs, including proofs of surprising 
results, such as that given in Example 4. 


Show that for every integer n there is a multiple of n that has only Os and Is in its decimal 
expansion. 


Solution: Letn be a positive integer. Consider the n + | integers 1, 11, 111,..., 11... 1 (where 
the last integer in this list is the integer with n + 1| Is in its decimal expansion). Note that there 
are n possible remainders when an integer is divided by n. Because there are n + | integers in 
this list, by the pigeonhole principle there must be two with the same remainder when divided 
by n. The larger of these integers less the smaller one is a multiple of n, which has a decimal 
expansion consisting entirely of Os and Is. < 


The Generalized Pigeonhole Principle 


The pigeonhole principle states that there must be at least two objects in the same box when 
there are more objects than boxes. However, even more can be said when the number of objects 
exceeds a multiple of the number of boxes. For instance, among any set of 21 decimal digits 
there must be 3 that are the same. This follows because when 21 objects are distributed into 
10 boxes, one box must have more than 2 objects. 


THE GENERALIZED PIGEONHOLE PRINCIPLE If N objects are placed into k 
boxes, then there is at least one box containing at least [N/k] objects. 


Proof: We will use a proof by contraposition. Suppose that none of the boxes contains more 
than [N/k] — 1 objects. Then, the total number of objects is at most 


(aes 


where the inequality [N/k] < (N/k) + 1 has been used. This is a contradiction because there 
are a total of N objects. | 


A common type of problem asks for the minimum number of objects such that at least r 
of these objects must be in one of k boxes when these objects are distributed among the boxes. 
When we have WN objects, the generalized pigeonhole principle tells us there must be at least r 
objects in one of the boxes as long as [N/k] > r. The smallest integer N with N/k > r—1, 
namely, N = k(r — 1) + 1, is the smallest integer satisfying the inequality [N/k] > r. Could 
a smaller value of N suffice? The answer is no, because if we had k(r — 1) objects, we could 
put r — | of them in each of the k boxes and no box would have at least r objects. 

When thinking about problems of this type, it is useful to consider how you can avoid having 
at least r objects in one of the boxes as you add successive objects. To avoid adding a rth object 
to any box, you eventually end up with r — 1 objects in each box. There is no way to add the 
next object without putting an rth object in that box. 

Examples 5-8 illustrate how the generalized pigeonhole principle is applied. 


Among 100 people there are at least [100/12] = 9 who were born in the same month. < 
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EXAMPLE 6 
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EXAMPLE 7 


A standard deck of 52 
cards has 13 kinds of 
cards, with four cards of 
each of kind, one in each 
of the four suits, hearts, 
diamonds, spades, and 
clubs. 


EXAMPLE 8 


EXAMPLE 9 


What is the minimum number of students required in a discrete mathematics class to be sure 
that at least six will receive the same grade, if there are five possible grades, A, B, C, D, and F? 


Solution: The minimum number of students needed to ensure that at least six students receive 
the same grade is the smallest integer N such that [N/5] = 6. The smallest such integer is 
N =5-5+1 = 26. If you have only 25 students, it is possible for there to be five who have re- 
ceived each grade so that no six students have received the same grade. Thus, 26 is the minimum 
number of students needed to ensure that at least six students will receive the same grade. <4 


a) How many cards must be selected from a standard deck of 52 cards to guarantee that at least 
three cards of the same suit are chosen? 
b) How many must be selected to guarantee that at least three hearts are selected? 


Solution: a) Suppose there are four boxes, one for each suit, and as cards are selected they are 
placed in the box reserved for cards of that suit. Using the generalized pigeonhole principle, 
we see that if N cards are selected, there is at least one box containing at least [N/4] cards. 
Consequently, we know that at least three cards of one suit are selected if [N/4] > 3. The 
smallest integer N such that [NV /4] > 3 is N =2-4+1=9, so nine cards suffice. Note that 
if eight cards are selected, it is possible to have two cards of each suit, so more than eight cards 
are needed. Consequently, nine cards must be selected to guarantee that at least three cards of 
one suit are chosen. One good way to think about this is to note that after the eighth card is 
chosen, there is no way to avoid having a third card of some suit. 


b) We do not use the generalized pigeonhole principle to answer this question, because we want 
to make sure that there are three hearts, not just three cards of one suit. Note that in the worst 
case, we can select all the clubs, diamonds, and spades, 39 cards in all, before we select a single 
heart. The next three cards will be all hearts, so we may need to select 42 cards to get three 
hearts. < 


What is the least number of area codes needed to guarantee that the 25 million phones in a state 
can be assigned distinct 10-digit telephone numbers? (Assume that telephone numbers are of 
the form NX X-NXX-XXXX, where the first three digits form the area code, N represents a 
digit from 2 to 9 inclusive, and X represents any digit.) 


Solution: There are eight million different phone numbers of the form NX X-X XXX (as shown 
in Example 8 of Section 6.1). Hence, by the generalized pigeonhole principle, among 25 million 
telephones, at least [25,000,000/8,000,000] = 4 of them must have identical phone numbers. 
Hence, at least four area codes are required to ensure that all 10-digit numbers are different. 4 


Example 9, although not an application of the generalized pigeonhole principle, makes use 
of similar principles. 


Suppose that a computer science laboratory has 15 workstations and 10 servers. A cable can be 
used to directly connect a workstation to a server. For each server, only one direct connection to 
that server can be active at any time. We want to guarantee that at any time any set of 10 or fewer 
workstations can simultaneously access different servers via direct connections. Although we 
could do this by connecting every workstation directly to every server (using 150 connections), 
what is the minimum number of direct connections needed to achieve this goal? 


Solution: Suppose that we label the workstations W,, W2,...,Wi5 and the servers 
S1, S2,..., S19. Furthermore, suppose that we connect W; to S_ fork = 1,2,..., 10 and each 
of Wi1, Wi2, Wi3, Wi4, and Wj5 to all 10 servers. We have a total of 60 direct connections. 
Clearly any set of 10 or fewer workstations can simultaneously access different servers. We see 
this by noting that if workstation W; is included with 1 < j < 10, it can access server $;, and 
for each workstation Wx with k > 11 included, there must be a corresponding workstation W; 


EXAMPLE 10 


EXAMPLE 11 
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with 1 < j < 10 not included, so W; can access server S;. (This follows because there are at 
least as many available servers S; as there are workstations W; with 1 < j < 10 not included.) 

Now suppose there are fewer than 60 direct connections between workstations and servers. 
Then some server would be connected to at most [59/10] = 5 workstations. (If all servers were 
connected to at least six workstations, there would be at least 6 - 10 = 60 direct connections.) 
This means that the remaining nine servers are not enough to allow the other 10 workstations to 
simultaneously access different servers. Consequently, at least 60 direct connections are needed. 
It follows that 60 is the answer. < 


Some Elegant Applications of the Pigeonhole Principle 


In many interesting applications of the pigeonhole principle, the objects to be placed in boxes 
must be chosen in a clever way. A few such applications will be described here. 


During a month with 30 days, a baseball team plays at least one game a day, but no more 
than 45 games. Show that there must be a period of some number of consecutive days during 
which the team must play exactly 14 games. 


Solution: Let a; be the number of games played on or before the jth day of the month. Then 
a|, 42,..., 430 1S an increasing sequence of distinct positive integers, with 1 < a; < 45. More- 
over, qj + 14, a2 + 14, ..., a39 + 14 is also an increasing sequence of distinct positive integers, 
with 15 < aj + 14 < 59. 

The 60 positive integers a1, d2,..., 439, 4 + 14, a2 + 14, ..., a3q + 14 are all less than 
or equal to 59. Hence, by the pigeonhole principle two of these integers are equal. Because the 
integers aj, j = 1,2,..., 30 are all distinct and the integers aj + 14, j = 1,2,..., 30 are all 
distinct, there must be indices i and j with a; = a; + 14. This means that exactly 14 games 
were played from day j + | to day i. 


Show that among any 1 + | positive integers not exceeding 2n there must be an integer that 
divides one of the other integers. 


Solution: Write each of the n + | integers aj, a2, ..., n41 aS a power of 2 times an odd integer. 
In other words, let aj = of qj tor j = 1,2,...,n +1, where k; is a nonnegative integer and qj; 
is odd. The integers g1, q2,.--, Gn+1 are all odd positive integers less than 2n. Because there 
are only n odd positive integers less than 2n, it follows from the pigeonhole principle that two 
of the integers gj, 92, .. - ; ¢n+1 must be equal. Therefore, there are distinct integers i and j such 
that gi = qj. Let g be the common value of q; and q;. Then, a; = 2g and q= 2*ig. It follows 
that if k; < k;, then a; divides a;; while if k; > k;, then a; divides a;. < 


A clever application of the pigeonhole principle shows the existence of an increasing or a 
decreasing subsequence of a certain length in a sequence of distinct integers. We review some 
definitions before this application is presented. Suppose that a}, a2,...,ay is a sequence of 
real numbers. A subsequence of this sequence is a sequence of the form aj,, dj,,..., di,,, where 
1 <i) <in <-+-- <im < N. Hence, a subsequence is a sequence obtained from the original 
sequence by including some of the terms of the original sequence in their original order, and 
perhaps not including other terms. A sequence is called strictly increasing if each term is larger 
than the one that precedes it, and it is called strictly decreasing if each term is smaller than the 
one that precedes it. 


Every sequence of n? + 1 distinct real numbers contains a subsequence of length n + 1 that 
is either strictly increasing or strictly decreasing. 
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EXAMPLE 12 


Links 


EXAMPLE 13 


We give an example before presenting the proof of Theorem 3. 


The sequence 8, 11, 9, 1, 4, 6, 12, 10, 5, 7 contains 10 terms. Note that 10 = 32 + 1. There 
are four strictly increasing subsequences of length four, namely, 1, 4, 6, 12; 1, 4, 6, 7; 
1, 4, 6, 10; and 1, 4, 5, 7. There is also a strictly decreasing subsequence of length four, 
namely, 11, 9, 6, 5. 


The proof of the theorem will now be given. 


Proof: Let aj, d2,..-,Q,244 be a sequence of n2 + 1 distinct real numbers. Associate an ordered 
pair with each term of the sequence, namely, associate (iz, dg) to the term az, where ix is the 
length of the longest increasing subsequence starting at ag, and d; is the length of the longest 
decreasing subsequence starting at ax. 

Suppose that there are no increasing or decreasing subsequences of length n + 1. Then ix 
and d, are both positive integers less than or equal to n, fork = 1,2,...,n? + 1. Hence, by the 
product rule there are n” possible ordered pairs for (ix, dg). By the pigeonhole principle, two of 
these n? + 1 ordered pairs are equal. In other words, there exist terms a, and a;, with s <t 
such that i; = i; and d; = d;. We will show that this is impossible. Because the terms of the 
sequence are distinct, either a; < a; or ds > a;. If as < a;, then, because is = i;, an increasing 
subsequence of length 7, + 1 can be built starting at a;, by taking a, followed by an increasing 
subsequence of length i; beginning at a;. This is a contradiction. Similarly, if a; > a;, the same 
reasoning shows that d,; must be greater than d;, which is a contradiction. | 


The final example shows how the generalized pigeonhole principle can be applied to an im- 
portant part of combinatorics called Ramsey theory, after the English mathematician F. P. Ram- 
sey. In general, Ramsey theory deals with the distribution of subsets of elements of sets. 


Assume that in a group of six people, each pair of individuals consists of two friends or two 
enemies. Show that there are either three mutual friends or three mutual enemies in the group. 


Solution: Let A be one of the six people. Of the five other people in the group, there are either 
three or more who are friends of A, or three or more who are enemies of A. This follows from 
the generalized pigeonhole principle, because when five objects are divided into two sets, one 
of the sets has at least [5/2] = 3 elements. In the former case, suppose that B, C, and D are 
friends of A. If any two of these three individuals are friends, then these two and A form a group 
of three mutual friends. Otherwise, B, C, and D form a set of three mutual enemies. The proof 
in the latter case, when there are three or more enemies of A, proceeds in a similar manner. < 


The Ramsey number R(m, n), where m and n are positive integers greater than or equal 
to 2, denotes the minimum number of people at a party such that there are either m mutual friends 
or n mutual enemies, assuming that every pair of people at the party are friends or enemies. 
Example 13 shows that R(3, 3) < 6. We conclude that R(3, 3) = 6 because in a group of five 


FRANK PLUMPTON RAMSEY (1903-1930) Frank Plumpton Ramsey, son of the president of Magdalene 
College, Cambridge, was educated at Winchester and Trinity Colleges. After graduating in 1923, he was elected 
a fellow of King’s College, Cambridge, where he spent the remainder of his life. Ramsey made important 
contributions to mathematical logic. What we now call Ramsey theory began with his clever combinatorial 
arguments, published in the paper “On a Problem of Formal Logic.” Ramsey also made contributions to the 
mathematical theory of economics. He was noted as an excellent lecturer on the foundations of mathematics. 
According to one of his brothers, he was interested in almost everything, including English literature and politics. 
Ramsey was married and had two daughters. His death at the age of 26 resulting from chronic liver problems 
deprived the mathematical community and Cambridge University of a brilliant young scholar. 
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people where every two people are friends or enemies, there may not be three mutual friends or 
three mutual enemies (see Exercise 26). 

It is possible to prove some useful properties about Ramsey numbers, but for the most 
part it is difficult to find their exact values. Note that by symmetry it can be shown that 
R(m, n) = R(n, m) (see Exercise 30). We also have R(2, n) = n for every positive integern > 2 
(see Exercise 29). The exact values of only nine Ramsey numbers R(m, n) with 3 < m < n are 
known, including R(4, 4) = 18. Only bounds are known for many other Ramsey numbers, in- 
cluding R(5, 5), which is known to satisfy 43 < R(5,5) < 49. The reader interested in learning 
more about Ramsey numbers should consult [MiRo91] or [GrRoSp90]. 


Exercises 


*10. 


. Show that in any set of six classes, each meeting regu- 


larly once a week on a particular day of the week, there 
must be two that meet on the same day, assuming that no 
classes are held on weekends. 


. Show that if there are 30 students in a class, then at least 


two have last names that begin with the same letter. 


. A drawer contains a dozen brown socks and a dozen black 


socks, all unmatched. A man takes socks out at random 
in the dark. 


a) How many socks must he take out to be sure that he 
has at least two socks of the same color? 


b) How many socks must he take out to be sure that he 
has at least two black socks? 


. A bowl contains 10 red balls and 10 blue balls. A woman 


selects balls at random without looking at them. 


a) How many balls must she select to be sure of having 
at least three balls of the same color? 


b) How many balls must she select to be sure of having 
at least three blue balls? 


. Show that among any group of five (not necessarily con- 


secutive) integers, there are two with the same remainder 
when divided by 4. 


. Let d be a positive integer. Show that among any group of 


d + 1 (not necessarily consecutive) integers there are two 
with exactly the same remainder when they are divided 
by d. 


. Let n be a positive integer. Show that in any set of n 


consecutive integers there is exactly one divisible by n. 


. Show that if f is a function from S to T, where S and T 


are finite sets with |$| > |T|, then there are elements s1 
and s2 in S such that f(s;) = f(s2), or in other words, f 
is not one-to-one. 


. What is the minimum number of students, each of whom 


comes from one of the 50 states, who must be enrolled in 
a university to guarantee that there are at least 100 who 
come from the same state? 


Let (x;, yj),¢ = 1, 2, 3, 4, 5, bea set of five distinct points 
with integer coordinates in the xy plane. Show that the 
midpoint of the line joining at least one pair of these 
points has integer coordinates. 


*11 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


. Let (xj, yj, Zi),1 = 1, 2,3, 4,5, 6, 7, 8, 9, be a set of nine 
distinct points with integer coordinates in xyz space. Show 
that the midpoint of at least one pair of these points has 
integer coordinates. 


How many ordered pairs of integers (a,b) are 
needed to guarantee that there are two ordered pairs 
(a,, b,) and (az, b2) such that aj mod 5 = az mod 5 
and b; mod 5 = b2 mod 5? 


a) Show that if five integers are selected from the first 
eight positive integers, there must be a pair of these 
integers with a sum equal to 9. 

b) Is the conclusion in part (a) true if four integers are 
selected rather than five? 


a) Show that if seven integers are selected from the first 
10 positive integers, there must be at least two pairs 
of these integers with the sum 11. 

b) Is the conclusion in part (a) true if six integers are 
selected rather than seven? 


How many numbers must be selected from the set 
{1, 2, 3, 4, 5, 6} to guarantee that at least one pair of these 
numbers add up to 7? 


How many numbers must be selected from the set 
{1, 3,5, 7,9, 11, 13, 15} to guarantee that at least one pair 
of these numbers add up to 16? 


A company stores products in a warehouse. Storage bins 
in this warehouse are specified by their aisle, location 
in the aisle, and shelf. There are 50 aisles, 85 horizontal 
locations in each aisle, and 5 shelves throughout the ware- 
house. What is the least number of products the company 
can have so that at least two products must be stored in 
the same bin? 


Suppose that there are nine students in a discrete mathe- 

matics class at a small college. 

a) Show that the class must have at least five male stu- 
dents or at least five female students. 

b) Show that the class must have at least three male stu- 
dents or at least seven female students. 

Suppose that every student in a discrete mathematics class 

of 25 students is a freshman, a sophomore, or a junior. 

a) Show that there are at least nine freshmen, at least 
nine sophomores, or at least nine juniors in the class. 
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20. 


21. 


22. 


* 23. 


** 24, 


* 25, 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


b) Show that there are either at least three freshmen, at 
least 19 sophomores, or at least five juniors in the 
class. 


Find an increasing subsequence of maximal length and 
a decreasing subsequence of maximal length in the se- 
quence 22, 5, 7, 2, 23, 10, 15, 21, 3, 17. 

Construct a sequence of 16 positive integers that has no 
increasing or decreasing subsequence of five terms. 


Show that if there are 101 people of different heights 
standing in a line, it is possible to find 11 people in the 
order they are standing in the line with heights that are 
either increasing or decreasing. 


Show that whenever 25 girls and 25 boys are seated 
around a circular table there is always a person both of 
whose neighbors are boys. 


Suppose that 21 girls and 21 boys enter a mathemat- 
ics competition. Furthermore, suppose that each entrant 
solves at most six questions, and for every boy-girl pair, 
there is at least one question that they both solved. Show 
that there is a question that was solved by at least three 
girls and at least three boys. 


Describe an algorithm in pseudocode for producing the 
largest increasing or decreasing subsequence of a se- 
quence of distinct integers. 


Show that in a group of five people (where any two people 
are either friends or enemies), there are not necessarily 
three mutual friends or three mutual enemies. 


Show that in a group of 10 people (where any two people 
are either friends or enemies), there are either three mu- 
tual friends or four mutual enemies, and there are either 
three mutual enemies or four mutual friends. 


Use Exercise 27 to show that among any group of 20 
people (where any two people are either friends or ene- 
mies), there are either four mutual friends or four mutual 
enemies. 


Show that if n is an integer with n > 2, then the Ramsey 
number R(2,n) equals n. (Recall that Ramsey numbers 
were discussed after Example 13 in Section 6.2.) 


Show that if m and n are integers with m > 2 andn > 2, 
then the Ramsey numbers R(m, n) and R(n, m) are equal. 
(Recall that Ramsey numbers were discussed after Exam- 
ple 13 in Section 6.2.) 


Show that there are at least six people in California (pop- 
ulation: 37 million) with the same three initials who were 
born on the same day of the year (but not necessarily in 
the same year). Assume that everyone has three initials. 


Show that if there are 100,000,000 wage earners in the 
United States who earn less than 1,000,000 dollars (but 
at least a penny), then there are two who earned exactly 
the same amount of money, to the penny, last year. 


In the 17th century, there were more than 800,000 inhab- 
itants of Paris. At the time, it was believed that no one had 
more than 200,000 hairs on their head. Assuming these 
numbers are correct and that everyone has at least one hair 
on their head (that is, no one is completely bald), use the 
pigeonhole principle to show, as the French writer Pierre 


34. 


35. 


36. 


37. 


38. 


39. 


* 40. 


41. 


* 42, 


43. 


44. 


Nicole did, that there had to be two Parisians with the 
same number of hairs on their heads. Then use the gener- 
alized pigeonhole principle to show that there had to be 
at least five Parisians at that time with the same number 
of hairs on their heads. 


Assuming that no one has more than 1,000,000 hairs on 
the head of any person and that the population of New 
York City was 8,008,278 in 2010, show there had to be at 
least nine people in New York City in 2010 with the same 
number of hairs on their heads. 


There are 38 different time periods during which classes 
at a university can be scheduled. If there are 677 different 
classes, how many different rooms will be needed? 


A computer network consists of six computers. Each com- 
puter is directly connected to at least one of the other com- 
puters. Show that there are at least two computers in the 
network that are directly connected to the same number 
of other computers. 


Acomputer network consists of six computers. Each com- 
puter is directly connected to zero or more of the other 
computers. Show that there are at least two computers in 
the network that are directly connected to the same num- 
ber of other computers. [Hint: It is impossible to have 
a computer linked to none of the others and a computer 
linked to all the others. ] 


Find the least number of cables required to connect eight 
computers to four printers to guarantee that for every 
choice of four of the eight computers, these four com- 
puters can directly access four different printers. Justify 
your answer. 


Find the least number of cables required to connect 100 
computers to 20 printers to guarantee that 2every subset 
of 20 computers can directly access 20 different printers. 
(Here, the assumptions about cables and computers are 
the same as in Example 9.) Justify your answer. 


Prove that at a party where there are at least two people, 
there are two people who know the same number of other 
people there. 


An arm wrestler is the champion for a period of 75 hours. 
(Here, by an hour, we mean a period starting from an 
exact hour, such as | P.M., until the next hour.) The arm 
wrestler had at least one match an hour, but no more than 
125 total matches. Show that there is a period of consec- 
utive hours during which the arm wrestler had exactly 24 
matches. 


Is the statement in Exercise 41 true if 24 is replaced by 

a) 2? b) 23? c) 25? d) 30? 
Show thatif f is afunction from S to T, where S and T are 
nonempty finite sets andm = [|S|/|T|], then there are at 
least m elements of S mapped to the same value of T. That 
is, show that there are distinct elements 51, 52,..., S of S 


such that f(s,) = f(s2) = +--+: = f(5m). 


There are 51 houses on a street. Each house has an address 
between 1000 and 1099, inclusive. Show that at least two 
houses have addresses that are consecutive integers. 


#45, 


46. 


Let x be an irrational number. Show that for some positive 
integer j not exceeding the positive integer n, the absolute 
value of the difference between jx and the nearest integer 
to jx is less than 1/n. 

Let n1,n2,...,n; be positive integers. Show that if 
ny+tno+---+n;—t+1 objects are placed into 
t boxes, then for some i, i = 1, 2,..., t, the ith box con- 
tains at least n; objects. 


*47, An alternative proof of Theorem 3 based on the general- 


Permutations and Combinations 


ized pigeonhole principle is outlined in this exercise. The 
notation used is the same as that used in the proof in the 
text. 
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a) Assume that i, < nfork =1,2,..., n* + 1. Use the 


b) 


c) 


generalized pigeonhole principle to show that there 
are n + | terms dg,, Qk, -.-, Ak, ,, With iz, =i, = 
++ = ig,,,, where | < ky < ky <+-+ < kyy1. 


Show that ag, > Aka) for 7 = 1,2,...,n.[Hint: As- 
sume that ak, < Akjy> and show that this implies that 
ik; > iki: which is a contradiction. ] 


Use parts (a) and (b) to show that if there is no increas- 
ing subsequence of length n + 1, then there must be 
a decreasing subsequence of this length. 


Introduction 


Many counting problems can be solved by finding the number of ways to arrange a specified 
number of distinct elements of a set of a particular size, where the order of these elements 
matters. Many other counting problems can be solved by finding the number of ways to select 
a particular number of elements from a set of a particular size, where the order of the elements 
selected does not matter. For example, in how many ways can we select three students from a 
group of five students to stand in line for a picture? How many different committees of three 
students can be formed from a group of four students? In this section we will develop methods 


to answer questions such as these. 


Permutations 


We begin by solving the first question posed in the introduction to this section, as well as related 


questions. 


EXAMPLE 1 


In how many ways can we select three students from a group of five students to stand in line for 


a picture? In how many ways can we arrange all five of these students in a line for a picture? 


Extra po 
Examples 2 


Solution: First, note that the order in which we select the students matters. There are five ways 
to select the first student to stand at the start of the line. Once this student has been selected, 


there are four ways to select the second student in the line. After the first and second students 
have been selected, there are three ways to select the third student in the line. By the product 
tule, there are 5 - 4 - 3 = 60 ways to select three students from a group of five students to stand 


in line for a picture. 


To arrange all five students in a line for a picture, we select the first student in five ways, 
the second in four ways, the third in three ways, the fourth in two ways, and the fifth in one 
way. Consequently, there are 5-4-3 -2-1 = 120 ways to arrange all five students in a line for 


a picture. 


< 


Example | illustrates how ordered arrangements of distinct objects can be counted. This leads 


to some terminology. 


A permutation of a set of distinct objects is an ordered arrangement of these objects. 
e We also are interested in ordered arrangements of some of the elements of a set. An ordered 


Links ix 


arrangement of r elements of a set is called an r-permutation. 
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EXAMPLE 2 


EXAMPLE 3 


THEOREM 1 


COROLLARY 1 


Let S = {1, 2, 3}. The ordered arrangement 3, 1, 2 is a permutation of S. The ordered arrangement 
3, 2 is a 2-permutation of S. 


The number of r-permutations of a set with n elements is denoted by P(n,r). We can find 
P(n,r) using the product rule. 


Let S={a,b,c}. The 2-permutations of S are the ordered arrangements 
a,b;a,c;b,a;b,c;c,a; and c,b. Consequently, there are six 2-permutations of this set 
with three elements. There are always six 2-permutations of a set with three elements. There 
are three ways to choose the first element of the arrangement. There are two ways to choose the 
second element of the arrangement, because it must be different from the first element. Hence, 
by the product rule, we see that P(3, 2) = 3-2 = 6. the first element. By the product rule, it 
follows that P(3,2) =3-2=6. < 


We now use the product rule to find a formula for P(n, r) whenever n and r are positive integers 
with 1 <r <n. 


If n is a positive integer and r is an integer with 1 < r <n, then there are 
P(n,r) =n(n—1)(n—2)---(2—rt+1) 


r-permutations of a set with n distinct elements. 


Proof: We will use the product rule to prove that this formula is correct. The first element of the 
permutation can be chosen in n ways because there are n elements in the set. There are n — | 
ways to choose the second element of the permutation, because there are n — | elements left 
in the set after using the element picked for the first position. Similarly, there are n — 2 ways 
to choose the third element, and so on, until there are exactly n — (r — 1) =n —r +1 ways to 
choose the rth element. Consequently, by the product rule, there are 


n(n — 1)(n—2)---(2-—r+1) 
r-permutations of the set. | 


Note that P(n, 0) = 1 whenever n is a nonnegative integer because there is exactly one 
way to order zero elements. That is, there is exactly one list with no elements in it, namely the 
empty list. 

We now state a useful corollary of Theorem 1. 


n! 


(n—r)! 


If n and r are integers with 0 < r <n, then P(n,r) = 


Proof: When n and r are integers with 1 < r <n, by Theorem | we have 


! 
PON =s6=0G29 Heese 
(n—Tr)! 
n! n! 
Because H_ol == 1 whenever n is a nonnegative integer, we see that the formula 
n—O)! nn! 
! 
— also holds when r = 0. | 


P(n,r) = aor 


EXAMPLE 4 


EXAMPLE 5 


EXAMPLE 6 


EXAMPLE 7 


EXAMPLE 8 
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By Theorem 1 we know that if n is a positive integer, then P(n, n) = n!. We will illustrate 
this result with some examples. 


How many ways are there to select a first-prize winner, a second-prize winner, and a third-prize 
winner from 100 different people who have entered a contest? 


Solution: Because it matters which person wins which prize, the number of ways to pick the 
three prize winners is the number of ordered selections of three elements from a set of 100 
elements, that is, the number of 3-permutations of a set of 100 elements. Consequently, the 
answer is 


P(100, 3) = 100 - 99 - 98 = 970,200. < 


Suppose that there are eight runners in a race. The winner receives a gold medal, the second- 
place finisher receives a silver medal, and the third-place finisher receives a bronze medal. How 
many different ways are there to award these medals, if all possible outcomes of the race can 
occur and there are no ties? 


Solution: The number of different ways to award the medals is the number of 3-permutations 
of a set with eight elements. Hence, there are P(8, 3) = 8-7-6 = 336 possible ways to award 
the medals. < 


Suppose that a saleswoman has to visit eight different cities. She must begin her trip in a specified 
city, but she can visit the other seven cities in any order she wishes. How many possible orders 
can the saleswoman use when visiting these cities? 


Solution: The number of possible paths between the cities is the number of permutations of 
seven elements, because the first city is determined, but the remaining seven can be ordered 
arbitrarily. Consequently, there are 7! = 7-6-5-4-3-2-+1 = 5040 ways for the saleswoman 
to choose her tour. If, for instance, the saleswoman wishes to find the path between the cities 
with minimum distance, and she computes the total distance for each possible path, she must 
consider a total of 5040 paths! < 


How many permutations of the letters ABCDEFGH contain the string ABC ? 


Solution: Because the letters ABC must occur as a block, we can find the answer by finding the 
number of permutations of six objects, namely, the block ABC and the individual letters D, E, 
F, G, and H. Because these six objects can occur in any order, there are 6! = 720 permutations 
of the letters ABCDEFGH in which ABC occurs as a block. < 


Combinations 


We now turn our attention to counting unordered selections of objects. We begin by solving a 
question posed in the introduction to this section of the chapter. 


How many different committees of three students can be formed from a group of four students? 


Solution: To answer this question, we need only find the number of subsets with three elements 
from the set containing the four students. We see that there are four such subsets, one for 
each of the four students, because choosing three students is the same as choosing one of the 
four students to leave out of the group. This means that there are four ways to choose the 
three students for the committee, where the order in which these students are chosen does not 
matter. < 
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Links 


EXAMPLE 9 


EXAMPLE 10 


THEOREM 2 


Example 8 illustrates that many counting problems can be solved by finding the number of 
subsets of a particular size of a set with n elements, where n is a positive integer. 

Anr-combination of elements of a set is an unordered selection of r elements from the set. 
Thus, an r-combination is simply a subset of the set with r elements. 


Let S be the set {1, 2, 3, 4}. Then {1, 3, 4} is a 3-combination from S. (Note that {4, 1, 3} is the 
same 3-combination as {1, 3, 4}, because the order in which the elements of a set are listed does 
not matter.) < 


The number of r-combinations of a set with n distinct elements is denoted by C(n, r). Note 
that C(n,r) is also denoted by (") and is called a binomial coefficient. We will learn where 
this terminology comes from in Section 6.4. 

We see that C(4, 2) = 6, because the 2-combinations of {a, b, c, d} are the six subsets {a, b}, 
{a, c}, {a, d}, {b, c}, {b, d}, and {c, d}. < 


We can determine the number of r-combinations of a set with n elements using the formula 
for the number of r-permutations of a set. To do this, note that the r-permutations of a set can be 
obtained by first forming r-combinations and then ordering the elements in these combinations. 
The proof of Theorem 2, which gives the value of C(n, r), is based on this observation. 


The number of r-combinations of a set with n elements, where n is a nonnegative integer and 
r is an integer with O < r < n, equals 


n! 
CT ri(n—r)! 


Proof: The P(n,r) r-permutations of the set can be obtained by forming the C(n,1r) 
r-combinations of the set, and then ordering the elements in each r-combination, which can be 
done in P(r, r) ways. Consequently, by the product rule, 


P(n,r) =C(n,r)- P(r, r). 
This implies that 


P(n,r) | n\/(n—r)! _ n! 
Pir) rl/r—n! ria—n! 


C(n,r) = 


We can also use the division rule for counting to construct a proof of this theorem. Because the 
order of elements in a combination does not matter and there are P(r, r) ways to order r elements 
in an r-combination of n elements, each of the C(n, r) r-combinations of a set with n elements 


corresponds to exactly P(r, r) r-permutations. Hence, by the division rule, C(n, r) = ane ‘ 
which implies as before that C(n, r) = Ce | 


The formula in Theorem 2, although explicit, is not helpful when C(n, r) is computed for 
large values of n and r. The reasons are that it is practical to compute exact values of factorials 
exactly only for small integer values, and when floating point arithmetic is used, the formula in 
Theorem 2 may produce a value that is not an integer. When computing C(n, r), first note that 
when we cancel out (n — r)! from the numerator and denominator of the expression for C(n, r) 
in Theorem 2, we obtain 


n! _ = Dee Gere) 


Grea rl 


EXAMPLE 11 


COROLLARY 2 
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Consequently, to compute C(n, r) you can cancel out all the terms in the larger factorial in the 
denominator from the numerator and denominator, then multiply all the terms that do not cancel 
in the numerator and finally divide by the smaller factorial in the denominator. [When doing 
this calculation by hand, instead of by machine, it is also worthwhile to factor out common 
factors in the numerator n(n — 1)--- (nm —r-+ 1) and in the denominator r!.] Note that many 
calculators have a built-in function for C(n, r) that can be used for relatively small values of n 
and r and many computational programs can be used to find C(n, r). [Such functions may be 
called choose(n, k) or binom(n, k)]. 

Example 11 illustrates how C(n, k) is computed when k is relatively small compared to n 
and when k is close to n. It also illustrates a key identity enjoyed by the numbers C(n, k). 


How many poker hands of five cards can be dealt from a standard deck of 52 cards? Also, how 


many ways are there to select 47 cards from a standard deck of 52 cards? 


Solution: Because the order in which the five cards are dealt from a deck of 52 cards does not 
matter, there are 

52! 
5!47! 


different hands of five cards that can be dealt. To compute the value of C (52, 5), first divide the 
numerator and denominator by 47! to obtain 


C(52,5) = 


§2-51-50-49- 48 
5.439961 


C (52,5) = 


This expression can be simplified by first dividing the factor 5 in the denominator into the 
factor SO in the numerator to obtain a factor 10 in the numerator, then dividing the factor 4 in the 
denominator into the factor 48 in the numerator to obtain a factor of 12 in the numerator, then 
dividing the factor 3 in the denominator into the factor 51 in the numerator to obtain a factor 
of 17 in the numerator, and finally, dividing the factor 2 in the denominator into the factor 52 in 
the numerator to obtain a factor of 26 in the numerator. We find that 


C(52,5) = 26-17- 10-49-12 = 2,598,960. 


Consequently, there are 2,598,960 different poker hands of five cards that can be dealt from a 
standard deck of 52 cards. 


Note that there are 
C(52, 47) = ae 
ee ATN5! 


different ways to select 47 cards from a standard deck of 52 cards. We do not need to compute 
this value because C(52, 47) = C(52, 5). (Only the order of the factors 5! and 47! is different 
in the denominators in the formulae for these quantities.) It follows that there are also 2,598,960 
different ways to select 47 cards from a standard deck of 52 cards. < 


In Example 11 we observed that C(52,5) = C(52, 47). This is a special case of the useful 
identity for the number of r-combinations of a set given in Corollary 2. 


Let n and r be nonnegative integers with r < n. Then C(n,r) = C(n,n —71r). 


Proof: From Theorem 2 it follows that 


n! 
LT) = a 


412 6/ Counting 


DEFINITION 1 


Combinatorial proofs 
are almost always much 
shorter and provide more 
insights than proofs 
based on algebraic 
manipulation. 


EXAMPLE 12 


Extra p> 
Examples 2 


EXAMPLE 13 


and 


n! n! 
= (n—r)![n—-(n—r)]! = (n—ry)ir! 


C(n,n—r) 


Hence, C(n,r) = C(n,n —r). <q 


We can also prove Corollary 2 without relying on algebraic manipulation. Instead, we can 
use a combinatorial proof. We describe this important type of proof in Definition 1. 


A combinatorial proof of an identity is a proof that uses counting arguments to prove that 
both sides of the identity count the same objects but in different ways or a proof that is based 
on showing that there is a bijection between the sets of objects counted by the two sides of 
the identity. These two types of proofs are called double counting proofs and bijective proofs, 
respectively. 


Many identities involving binomial coefficients can be proved using combinatorial proofs. We 
now show how to prove Corollary 2 using a combinatorial proof. We will provide both a double 
counting proof and a bijective proof, both based on the same basic idea. 


Proof: We will use a bijective proof to show that C(n,r) = C(n,n —r) for all integers n 
andr with 0 < r <n. Suppose that S is a set with n elements. The function that maps a subset 
A of S to A is a bijection between subsets of S with r elements and subsets with n — r elements 
(as the reader should verify). The identity C(n, r) = C(n, n — r) follows because when there 
is a bijection between two finite sets, the two sets must have the same number of elements. 
Alternatively, we can reformulate this argument as a double counting proof. By definition, 
the number of subsets of S with r elements equals C(n,r). But each subset A of S is also 
determined by specifying which elements are not in A, and so are in A. Because the complement 
of a subset of S with r elements has n — r elements, there are also C(n, n — r) subsets of S 
with r elements. It follows that C(n,r) = C(n,n —1r). <q 


How many ways are there to select five players from a 10-member tennis team to make a trip to 
a match at another school? 


Solution: The answer is given by the number of 5-combinations of a set with 10 elements. By 
Theorem 2, the number of such combinations is 


10! 
C(O, 5) = oe = 252. < 


A group of 30 people have been trained as astronauts to go on the first mission to Mars. How 
many ways are there to select a crew of six people to go on this mission (assuming that all crew 
members have the same job)? 


Solution: The number of ways to select a crew of six from the pool of 30 people is the number 
of 6-combinations of a set with 30 elements, because the order in which these people are chosen 
does not matter. By Theorem 2, the number of such combinations is 


30! 30-29-28 - 27-26-25 


C(30, 6) = = 
( 6! 24! 6-5-4-3-2-1 


= 593,775. 


EXAMPLE 14 
EXAMPLE 15 
9! 
C(9, 3)-C(11, 4) = na 
Exercises 


10. 


11. 
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How many bit strings of length n contain exactly r 1s? 


Solution: The positions of r Is in a bit string of length n form an r-combination of the set 
{1, 2,3,...,}. Hence, there are C(n, r) bit strings of length n that contain exactlyr 1s. <4 


Suppose that there are 9 faculty members in the mathematics department and 11 in the computer 
science department. How many ways are there to select a committee to develop a discrete 
mathematics course at a school if the committee is to consist of three faculty members from the 
mathematics department and four from the computer science department? 


Solution: By the product rule, the answer is the product of the number of 3-combinations of 
a set with nine elements and the number of 4-combinations of a set with 11 elements. By 


Theorem 2, the number of ways to select the committee is 


. List all the permutations of {a, b, c}. 
. How many different permutations are there of the set 


{a, b,c, d,e, f, g}? 


. How many permutations of {a, b, c,d, e, f, g} end with 


a? 


. Let S = {1, 2, 3, 4, 5}. 


a) List all the 3-permutations of S. 
b) List all the 3-combinations of S. 


. Find the value of each of these quantities. 


12. 


13. 


14. 


= 84. 330 = 27,720. 


How many bit strings of length 12 contain 

a) exactly three 1s? 

b) at most three 1s? 

c) at least three 1s? 

d) an equal number of Os and 1s? 

A group contains n men and n women. How many ways 
are there to arrange these people in a row if the men and 
women alternate? 

In how many ways can a set of two positive integers less 
than 100 be chosen? 


a) P(6, 3) b) P(6,5) 15. In how many ways can a set of five letters be selected 

c) P(8, 1) d) P(8,5) from the English alphabet? 

e) P(8,8) f) P(10, 9) 16. How many subsets with an odd number of elements does 
. Find the value of each of these quantities. a set with 10 elements have? 

a) C(5, 1) b) C(5, 3) 17. How many subsets with more than two elements does a 

c) C(8, 4) d) C(8, 8) set with 100 elements have? 

e) C(8, 0) f) C(12, 6) 18. A coin is flipped eight times where each flip comes up 


. Find the number of 5-permutations of a set with nine el- 


ements. 


. In how many different orders can five runners finish a 


race if no ties are allowed? 


. How many possibilities are there for the win, place, and 


show (first, second, and third) positions in a horse race 
with 12 horses if all orders of finish are possible? 

There are six different candidates for governor of a state. 
In how many different orders can the names of the can- 
didates be printed on a ballot? 

How many bit strings of length 10 contain 

a) exactly four 1s? 

b) at most four 1s? 

c) at least four 1s? 

d) an equal number of Os and 1s? 


19. 


20. 


either heads or tails. How many possible outcomes 


a) are there in total? 

b) contain exactly three heads? 

c) contain at least three heads? 

d) contain the same number of heads and tails? 
A coin is flipped 10 times where each flip comes up either 
heads or tails. How many possible outcomes 

a) are there in total? 

b) contain exactly two heads? 

c) contain at most three tails? 

d) contain the same number of heads and tails? 
How many bit strings of length 10 have 

a) exactly three Os? 

b) more Os than 1s? 

c) at least seven 1s? 

d) at least three 1s? 


414 6/Counting 


21. 


22. 


23. 


24. 


25. 


26. 


27. 


How many permutations of the letters ABCDEFG con- 
tain 

a) the string BCD? 

b) the string CFGA? 

c) the strings BA and GF? 

d) the strings ABC and DE? 

e) the strings ABC and CDE? 

f) the strings CBA and BED? 

How many permutations of the letters ABCDEFGH con- 
tain 

a) the string ED? 

b) the string CDE? 

c) the strings BA and FGH? 

d) the strings AB, DE, and GH? 

e) the strings CAB and BED? 

f) the strings BCA and ABF? 

How many ways are there for eight men and five women 
to stand in a line so that no two women stand next to each 
other? [Hint: First position the men and then consider 
possible positions for the women.] 

How many ways are there for 10 women and six men 
to stand in a line so that no two men stand next to each 
other? [Hint: First position the women and then consider 
possible positions for the men.] 

One hundred tickets, numbered 1, 2, 3, ..., 100, are sold 
to 100 different people for a drawing. Four different prizes 
are awarded, including a grand prize (a trip to Tahiti). How 
many ways are there to award the prizes if 


a) there are no restrictions? 

b) the person holding ticket 47 wins the grand prize? 

c) the person holding ticket 47 wins one of the prizes? 

d) the person holding ticket 47 does not win a prize? 

e) the people holding tickets 19 and 47 both win prizes? 

f) the people holding tickets 19, 47, and 73 all win 
prizes? 

g) the people holding tickets 19, 47, 73, and 97 all win 
prizes? 

h) none of the people holding tickets 19, 47, 73, and 97 
wins a prize? 

i) the grand prize winner is a person holding ticket 19, 
47, 73, or 97? 

j)_ the people holding tickets 19 and 47 win prizes, but 
the people holding tickets 73 and 97 do not win prizes? 


Thirteen people on a softball team show up for a game. 


a) How many ways are there to choose 10 players to take 
the field? 

b) How many ways are there to assign the 10 positions 
by selecting players from the 13 people who show up? 

c) Ofthe 13 people who show up, three are women. How 
many ways are there to choose 10 players to take the 
field if at least one of these players must be a woman? 


A club has 25 members. 


a) How many ways are there to choose four members of 
the club to serve on an executive committee? 

b) How many ways are there to choose a president, vice 
president, secretary, and treasurer of the club, where 
no person can hold more than one office? 


28 


* 29, 


. A professor writes 40 discrete mathematics true/false 
questions. Of the statements in these questions, 17 are 
true. If the questions can be positioned in any order, how 
many different answer keys are possible? 


How many 4-permutations of the positive integers not ex- 

ceeding 100 contain three consecutive integers k, k + 1, 

k + 2, in the correct order 

a) where these consecutive integers can perhaps be sep- 
arated by other integers in the permutation? 


b) where they are in consecutive positions in the permu- 


tation? 

30. Seven women and nine men are on the faculty in the 
mathematics department at a school. 

a) How many ways are there to select a committee of 
five members of the department if at least one woman 
must be on the committee? 

b) How many ways are there to select a committee of 
five members of the department if at least one woman 
and at least one man must be on the committee? 

31. The English alphabet contains 21 consonants and five 
vowels. How many strings of six lowercase letters of the 
English alphabet contain 
a) exactly one vowel? 

b) exactly two vowels? 

c) at least one vowel? 

d) at least two vowels? 

32. How many strings of six lowercase letters from the En- 
glish alphabet contain 
a) the letter a? 

b) the letters a and b? 

c) the letters a and b in consecutive positions with a 
preceding b, with all the letters distinct? 

d) the letters a and b, where a is somewhere to the left 
of b in the string, with all the letters distinct? 

33. Suppose that a department contains 10 men and 15 
women. How many ways are there to form a commit- 
tee with six members if it must have the same number of 
men and women? 

34. Suppose that a department contains 10 men and 15 
women. How many ways are there to form a commit- 
tee with six members if it must have more women than 
men? 

35. How many bit strings contain exactly eight Os and 10 Is 
if every 0 must be immediately followed by a 1? 

36. How many bit strings contain exactly five Os and 14 1s if 
every 0 must be immediately followed by two 1s? 

37. How many bit strings of length 10 contain at least three 
1s and at least three Os? 

38. How many ways are there to select 12 countries in the 


United Nations to serve on a council if 3 are selected 
from a block of 45, 4 are selected from a block of 57, and 
the others are selected from the remaining 69 countries? 


39. 


How many license plates consisting of three letters fol- 
lowed by three digits contain no letter or digit twice? 


A circular r-permutation of n people is a seating of r of 
these n people around a circular table, where seatings are con- 
sidered to be the same if they can be obtained from each other 
by rotating the table. 


40. 
41. 


42. 


43. 


*44, 


*45, 


Find the number of circular 3-permutations of 5 people. 


Find a formula for the number of circular r-permutations 
of n people. 


Find a formula for the number of ways to seat r of n people 
around a circular table, where seatings are considered the 
same if every person has the same two neighbors without 
regard to which side these neighbors are sitting on. 


How many ways are there for a horse race with three 
horses to finish if ties are possible? [Note: Two or three 
horses may tie.] 


How many ways are there for a horse race with four horses 
to finish if ties are possible? [Note: Any number of the 
four horses may tie.) 


There are six runners in the 100-yard dash. How many 
ways are there for three medals to be awarded if ties 
are possible? (The runner or runners who finish with the 
fastest time receive gold medals, the runner or runners 
who finish with exactly one runner ahead receive silver 


* 46. 
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medals, and the runner or runners who finish with exactly 
two runners ahead receive bronze medals.) 


This procedure is used to break ties in games in the cham- 
pionship round of the World Cup soccer tournament. Each 
team selects five players in a prescribed order. Each of 
these players takes a penalty kick, with a player from the 
first team followed by a player from the second team and 
so on, following the order of players specified. If the score 
is still tied at the end of the 10 penalty kicks, this proce- 
dure is repeated. If the score is still tied after 20 penalty 
kicks, a sudden-death shootout occurs, with the first team 
scoring an unanswered goal victorious. 


a) How many different scoring scenarios are possible if 
the game is settled in the first round of 10 penalty 
kicks, where the round ends once it is impossible for 
a team to equal the number of goals scored by the 
other team? 

b) How many different scoring scenarios for the first 
and second groups of penalty kicks are possible if 
the game is settled in the second round of 10 penalty 
kicks? 

c) How many scoring scenarios are possible for the full 
set of penalty kicks if the game is settled with no more 
than 10 total additional kicks after the two rounds of 
five kicks for each team? 


6.4 Binomial Coefficients and Identities 


Links 


EXAMPLE 1 


As we remarked in Section 6.3, the number of r-combinations from a set with n elements is 
often denoted by (). This number is also called a binomial coefficient because these numbers 
occur as coefficients in the expansion of powers of binomial expressions such as (a + b)". We 
will discuss the binomial theorem, which gives a power of a binomial expression as a sum of 
terms involving binomial coefficients. We will prove this theorem using a combinatorial proof. 
We will also show how combinatorial proofs can be used to establish some of the many different 


identities that express relationships among binomial coefficients. 


The Binomial Theorem 


The binomial theorem gives the coefficients of the expansion of powers of binomial expressions. 
A binomial expression is simply the sum of two terms, such as x + y. (The terms can be products 
of constants and variables, but that does not concern us here.) 

Example | illustrates how the coefficients in a typical expansion can be found and prepares 
us for the statement of the binomial theorem. 


The expansion of (x + y)? can be found using combinatorial reasoning instead of multiplying 
the three terms out. When (x + y)3 = (x + y)(x + y)(x + y) is expanded, all products of a 
term in the first sum, a term in the second sum, and a term in the third sum are added. Terms of 
the form x3, x”y, xy”, and y? arise. To obtain a term of the form x3, an x must be chosen in 
each of the sums, and this can be done in only one way. Thus, the x? term in the product has 
a coefficient of 1. To obtain a term of the form x”y, an x must be chosen in two of the three 
sums (and consequently a y in the other sum). Hence, the number of such terms is the number 
of 2-combinations of three objects, namely, (). Similarly, the number of terms of the form xy” 
is the number of ways to pick one of the three sums to obtain an x (and consequently take a y 
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from each of the other two sums). This can be done in (?) ways. Finally, the only way to obtain 
a y> term is to choose the y for each of the three sums in the product, and this can be done in 
exactly one way. Consequently, it follows that 


+yP=+yW+y+y) = Ox+xytyx+yy)(xt+y) 
SAXXX + AXXY + XYX + XY + YXX + yxy + yyxX + yyy 
= x3 + 3x?y + 3xy? + y?. ca 


We now state the binomial theorem. 


THEOREM1 THEBINOMIALTHEOREM Letx and y be variables, and letn be a nonnegative integer. 
Then 


n n 3 . n n n n 
Cy) ) (Ge = (ie + (Vey fees ( ee + ( yr 
j=o J ae n 


Proof: We use a combinatorial proof. The terms in the product when it is expanded are of 
the form x” / y/ for 7 =0,1,2,...,. To count the number of terms of the form x”-/ y/, 
note that to obtain such a term it is necessary to choose n — j xs from the n sums (so that the 


other j terms in the product are ys). Therefore, the coefficient of x”~/ y/ is (2 > which is 


equal to (4). This proves the theorem. se] 
Some computational uses of the binomial theorem are illustrated in Examples 2-4. 
EXAMPLE2_ What is the expansion of (x + y)*? 


Extra © Solution: From the binomial theorem it follows that 


Examples 4 


wate (ey 


j=0 


“(30 ()ere Geren 


= x44 4x3y + 6x7y? + 4xy? + yt. < 
EXAMPLE3_ What is the coefficient of x!*y!? in the expansion of (x + y)?>? 


Solution: From the binomial theorem it follows that this coefficient is 


25 25! 
a = $9 
(73) 13! 12! sacle < 


EXAMPLE 4 _ What is the coefficient of x!*y!? in the expansion of (2x — 3y)*>? 
Solution: First, note that this expression equals (2x + (—3 y))> . By the binomial theorem, we 


have 


S05 
(2x + (—3y))* =) Pen (-3y)/. 


j=0 
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Consequently, the coefficient of x!*y!> in the expansion is obtained when j = 13, namely, 
25 25! 
912(_3)13 _ 712313, 
a = 13! 12! < 


We can prove some useful identities using the binomial theorem, as Corollaries 1, 2, and 3 
demonstrate. 


COROLLARY1 Let be a nonnegative integer. Then 


(= 


k=0 


Proof: Using the binomial theorem with x = | and y = 1, we see that 


on — (1+ if _ 3 (;) pk yr-* = > a). 
k=0 


This is the desired result. <q 


There is also a nice combinatorial proof of Corollary 1, which we now present. 


Proof: A set with n elements has a total of 2” different subsets. Each subset has zero elements, 


one element, two elements, ..., or elements in it. There are i) subsets with zero elements, (i) 
subsets with one element, (5) subsets with two elements, ..., and (") subsets with n elements. 
Therefore, 


n 
> (;) 
k=0 k 
counts the total number of subsets of a set with n elements. By equating the two formulas we 
have for the number of subsets of a set with n elements, we see that 


COROLLARY 2 _ Letn bea positive integer. Then 


yew?) =p 
k= 0 k 


Proof: When we use the binomial theorem with x = —1 and y = 1, we see that 
n n n n 
0=0'= =| LD’ = _] kyn-k _ _| ke 
(-1) + 1) Li) 2G JOD 


This proves the corollary. 4 
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COROLLARY 3 


THEOREM 2 


Remark: Corollary 2 implies that 
=F 7 =F " spree ‘i =F i =F : + 
0 2 4 “a4 3 5 


Let n be a nonnegative integer. Then 


ss 2k © 3 


0) 


Proof: We recognize that the left-hand side of this formula is the expansion of (1 + 2)” provided 
by the binomial theorem. Therefore, by the binomial theorem, we see that 


(1+2)"= 3 (ee - Sy (72 
k=0 


k=0 


Pascal’s Identity and Triangle 


The binomial coefficients satisfy many different identities. We introduce one of the most im- 
portant of these now. 


PASCAL’S IDENTITY Let n and k be positive integers with n > k. Then 
n+1\ _ n s n 
k J \k-1 ey: 


Proof: We will use a combinatorial proof. Suppose that T is a set containing n + | elements. Let 
a be an element in 7, and let §S = T — {a}. Note that there are () subsets of T containing k 
elements. However, a subset of T with k elements either contains a together with k — 1 elements 


of S, or contains k elements of S$ and does not contain a. Because there are (a) subsets of 
k — 1 elements of S, there are (e) subsets of k elements of T that contain a. And there are 


G) subsets of k elements of T that do not contain a, because there are (7) subsets of k elements 
of S. Consequently, 


r)-G2 +: d 


Remark: It is also possible to prove this identity by algebraic manipulation from the formula 
for C) (see Exercise 19). 
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(3) 
(5) G) 14 
) G) @) By Pascal's identity 1201 
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(a) (b) 
FIGURE 1 Pascal’s Triangle. 


Remark: Pascal’s identity, together with the initial conditions (5) — (") = | for all integers n, 


can be used to recursively define binomial coefficients. This recursive definition is useful in the 
computation of binomial coefficients because only addition, and not multiplication, of integers 
is needed to use this recursive definition. 


Pascal’s identity is the basis for a geometric arrangement of the binomial coefficients in a 
triangle, as shown in Figure 1. 
The nth row in the triangle consists of the binomial coefficients 


(i). die ee 


This triangle is known as Pascal’s triangle. Pascal’s identity shows that when two adjacent 
binomial coefficients in this triangle are added, the binomial coefficient in the next row between 
these two coefficients is produced. 


BLAISE PASCAL (1623-1662) Blaise Pascal exhibited his talents at an early age, although his father, who 
had made discoveries in analytic geometry, kept mathematics books away from him to encourage other interests. 
At 16 Pascal discovered an important result concerning conic sections. At 18 he designed a calculating machine, 
which he built and sold. Pascal, along with Fermat, laid the foundations for the modern theory of probability. In 
this work, he made new discoveries concerning what is now called Pascal’s triangle. In 1654, Pascal abandoned 
his mathematical pursuits to devote himself to theology. After this, he returned to mathematics only once. One 
night, distracted by a severe toothache, he sought comfort by studying the mathematical properties of the cycloid. 
Miraculously, his pain subsided, which he took as a sign of divine approval of the study of mathematics. 
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THEOREM 3 


Links 


COROLLARY 4 


Other Identities Involving Binomial Coefficients 


We conclude this section with combinatorial proofs of two of the many identities enjoyed by 
the binomial coefficients. 


VANDERMONDE’S IDENTITY Let m, n, and 7 be nonnegative integers with r not 
exceeding either m orn. Then 


("*") -r(")() 


Remark: This identity was discovered by mathematician Alexandre-Théophile Vandermonde 
in the eighteenth century. 


Proof: Suppose that there are m items in one set and n items in a second set. Then the total 
number of ways to pick r elements from the union of these sets is (”" * ae 

Another way to pick r elements from the union is to pick k elements from the second set 
and then r — k elements from the first set, where k is an integer with O < k < r. Because there 


are 4) ways to choose k elements from the second set and (, m a) ways to choose r — k elements 


m 


=) (7) ways. Hence, the 


total number of ways to pick r elements from the union also equals )~y — (,.",) (Z)- 
ak =0 \r-) k 

We have found two expressions for the number of ways to pick r elements from the 

union of a set with m items and a set with n items. Equating them gives us Vandermonde’s 


identity. | 


from the first set, the product rule tells us that this can be done in ( 


Corollary 4 follows from Vandermonde’s identity. 


If n is a nonnegative integer, then 
G20) 
n ae k 


Proof: We use Vandermonde’s identity with m = r = n to obtain 


BP ero. 6) 


The last equality was obtained using the identity (7) = (," ;). 4 


Links 


ALEXANDRE-THEOPHILE VANDERMONDE (1735-1796) Because Alexandre-Théophile Vandermonde was a sickly child, his 
physician father directed him to a career in music. However, he later developed an interest in mathematics. His complete mathematical 
work consists of four papers published in 1771-1772. These papers include fundamental contributions on the roots of equations, on 
the theory of determinants, and on the knight’s tour problem (introduced in the exercises in Section 10.5). Vandermonde’s interest in 
mathematics lasted for only 2 years. Afterward, he published papers on harmony, experiments with cold, and the manufacture of steel. 
He also became interested in politics, joining the cause of the French revolution and holding several different positions in government. 
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We can prove combinatorial identities by counting bit strings with different properties, as 
the proof of Theorem 4 will demonstrate. 


THEOREM 4 


lei 


Proof: We use a combinatorial proof. By Example 14 in Section 6.3, the left-hand side, ( 


Let n and r be nonnegative integers with r < n. Then 


rps 


counts the bit strings of length n + | containing r + | ones. 

We show that the right-hand side counts the same objects by considering the cases corre- 
sponding to the possible locations of the final | in a string with r + | ones. This final one must 
occur at positionr + 1,r +2,..., orn + 1. Furthermore, if the last one is the kth bit there must 
be r ones among the first k — 1 positions. Consequently, by Example 14 in Section 6.3, there 
are er ') such bit strings. Summing over k withr + 1 < k <n +1, we find that there are 


n+1 


Py 


j=r 


(=z) 


bit strings of length n containing exactly r + 1 ones. (Note that the last step follows from the 
change of variables j = k — 1.) Because the left-hand side and the right-hand side count the 
same objects, they are equal. This completes the proof. | 


Exercises 


im) 


SN 


*11, 


12. 


. Find the expansion of (x + y)* 


a) using combinatorial reasoning, as in Example 1. 
b) using the binomial theorem. 


. Find the expansion of (x + y)> 


a) using combinatorial reasoning, as in Example 1. 
b) using the binomial theorem. 


. Find the expansion of (x + y)®. 
. Find the coefficient of x> y® in (x + y)}. 
. How many terms are there in the expansion of (x + y)! 


after like terms are collected? 


. What is the coefficient of x? in (1 + x)!!? 
. What is the coefficient of x? in (2 — x)!9? 
. What is the coefficient of x8 y? in the expansion of 


(3x + 2y)!79 


. What is the coefficient of x!°!y% in the expansion of 


(2x — 3y)20? 


. Give a formula for the coefficient of x* in the expansion 


of (x + 1/x)!©, where k is an integer. 

Give a formula for the coefficient of x* in the expansion 
of (x2 — 1/x)!, where k is an integer. 

The row of Pascal’s triangle containing the binomial co- 
efficients Cy, 0<k < 10, is: 


1 10 45 120 210 252 210 120 45 10 1 


Use Pascal’s identity to produce the row immediately fol- 
lowing this row in Pascal’s triangle. 


13. 


14. 


15. 


16. 


C="17, 


18. 


19. 
20. 


What is the row of Pascal’s triangle containing the bino- 

mial coefficients (), 0<k <9? 

Show that if n is a positive integer, then 1 = (5) < 

oS (1n/2)) = (mya) ae (nna) = (1) =1. 

Show that (1) < 2” for all positive integers n and all in- 

tegers kK withO <k <n. 

a) Use Exercise 14 and Corollary | to show that if n is 
an integer greater than 1, then (inj ) > 2"/n. 

b) Conclude from part (a) that if n is a positive integer, 
then (2") > 4" /2n. 

Show that if n and k are integers with 1 < k <n, then 

(7) < nk /2k-1, 

Suppose that b is an integer with b > 7. Use the bino- 

mial theorem and the appropriate row of Pascal’s triangle 

to find the base-b expansion of (1 ly [that is, the fourth 

power of the number (11), in base-b notation]. 

Prove Pascal’s identity, using the formula for (”). 


Suppose that k and n are integers with 1 < k <n. Prove 
the hexagon identity 


n-1 n n+1\)_ (n-1 n n+1 

k-1)\k+1 kk} \ kk JXR-1) Kk 417 
which relates terms in Pascal’s triangle that form a 
hexagon. 
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[27], 


22. 


23. 


24, 


25. 


* 26. 


* 27, 


28. 


* 29, 


* 30. 


31. 


* 32. 


Prove that if n and k are integers with 1 < k <n, then 

(i) = nai), 

a) using a combinatorial proof. [Hint: Show that the two 
sides of the identity count the number of ways to select 
a subset with k elements from a set with n elements 
and then an element of this subset. ] 

b) using an algebraic proof based on the formula for (7) 
given in Theorem 2 in Section 6.3. 

Prove the identity (”)({) = (7) Ca), whenever n, r, and 

k are nonnegative integers withr <n andk <r, 

a) using a combinatorial argument. 

b) using an argument based on the formula for the num- 
ber of r-combinations of a set with n elements. 

Show that if n and k are positive integers, then 


(“F')=@+0(,")) /k. 


Use this identity to construct an inductive definition of 

the binomial coefficients. 

Show that if p is a prime and k is an integer such that 
a p 

1 <k < p-—1, then p divides (°). 

Let n be a positive integer. Show that 


2n 2n 2n+2 
(, iv*G)=Gan)? 


Let n and k be integers with 1 < k <n. Show that 


OG )=Ca)e-C). 


Prove the hockeystick identity 


ee aC) 


whenever n and r are positive integers, 

a) using a combinatorial argument. 

b) using Pascal’s identity. 

Show that if n is a positive integer, then CG) =2(5) +n? 
a) using a combinatorial argument. 

b) by algebraic manipulation. 

Give a combinatorial proof that )>y_ , k(j) =22""!. 
[Hint: Count in two ways the number of ways to select a 
committee and to then select a leader of the committee. ] 


Give a combinatorial proof that }77_ tag =n(7"7}). 


[Hint: Count in two ways the number of ways to select a 
committee, with n members from a group of n mathemat- 
ics professors and n computer science professors, such 
that the chairperson of the committee is a mathematics 
professor. ] 

Show that a nonempty set has the same number of subsets 
with an odd number of elements as it does subsets with 
an even number of elements. 


Prove the binomial theorem using mathematical induc- 
tion. 


33. 


(0, 0) 


(0, 0) 


ime) 


36. 


37. 


38. 


* 39, 


4. 


In this exercise we will count the number of paths in the 
xy plane between the origin (0, 0) and point (m, n), where 
m and n are nonnegative integers, such that each path is 
made up of a series of steps, where each step is a move one 
unit to the right or a move one unit upward. (No moves to 
the left or downward are allowed.) Two such paths from 
(0, 0) to (5, 3) are illustrated here. 


(5, 3) 


(5, 3) 


a) Show that each path of the type described can be rep- 
resented by a bit string consisting of m Os and n 1s, 
where a 0 represents a move one unit to the right and 
a | represents a move one unit upward. 

b) Conclude from part (a) that there are nag paths of 
the desired type. 

Use Exercise 33 to give an alternative proof of Corollary 2 

in Section 6.3, which states that (7) = (,",) whenever k 

is an integer with 0 < k <n. [Hint: Consider the number 

of paths of the type described in Exercise 33 from (0, 0) 

to (n — k, k) and from (0, 0) to (k,n — k).] 


. Use Exercise 33 to prove Theorem 4. [Hint: Count the 


number of paths with n steps of the type described in Ex- 
ercise 33. Every such path must end at one of the points 
(n —k,k) fork =0,1,2,...,n.] 

Use Exercise 33 to prove Pascal’s identity. [Hint: Show 
that a path of the type described in Exercise 33 
from (0,0) to (n+1—k,k) passes through either 
(n+1—k,k —1) or (xn —k,k), but not through both.] 
Use Exercise 33 to prove the hockeystick identity from 
Exercise 27. [Hint: First, note that the number of 
paths from (0,0) to (n+ 1,r) equals ae +"). Sec- 
ond, count the number of paths by summing the num- 
ber of these paths that start by going k units upward for 
k=0,1,2,...,7r.] 

Give a combinatorial proof that if n is a positive inte- 
ger then )-7_ 9 k*() = n(n + 1)2”®. [Hint: Show that 
both sides count the ways to select a subset of a set of n ele- 
ments together with two not necessarily distinct elements 
from this subset. Furthermore, express the right-hand side 
as n(n — 1)2"-2 4+ n2"—-! J 

Determine a formula involving binomial coefficients for 
the nth term of a sequence if its initial terms are those 
listed. [Hint: Looking at Pascal’s triangle will be helpful. 
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Although infinitely many sequences start with a specified c) 1, 2,6, 20, 70, 252, 924, 3432, 12870, 48620, ... 

set of terms, each of the following lists is the start of a d) 1, 1, 2, 3, 6, 10, 20, 35, 70, 126,... 

sequence of the type desired.] e) 1,1, 1,3, 1,5, 15,35, 1,9,... 

a) 1, 3,6, 10, 15, 21, 28, 36, 45, 55, 66,... f) 1, 3, 15, 84, 495, 3003, 18564, 116280, 735471, 
b) 1, 4, 10, 20, 35, 56, 84, 120, 165, 220,... 4686825, 


Generalized Permutations and Combinations 


Links 


EXAMPLE 1 


THEOREM 1 


Introduction 


In many counting problems, elements may be used repeatedly. For instance, a letter or digit may 
be used more than once on a license plate. When a dozen donuts are selected, each variety can 
be chosen repeatedly. This contrasts with the counting problems discussed earlier in the chapter 
where we considered only permutations and combinations in which each item could be used at 
most once. In this section we will show how to solve counting problems where elements may 
be used more than once. 

Also, some counting problems involve indistinguishable elements. For instance, to count the 
number of ways the letters of the word SUCCESS can be rearranged, the placement of identical 
letters must be considered. This contrasts with the counting problems discussed earlier where all 
elements were considered distinguishable. In this section we will describe how to solve counting 
problems in which some elements are indistinguishable. 

Moreover, in this section we will explain how to solve another important class of counting 
problems, problems involving counting the ways distinguishable elements can be placed in 
boxes. An example of this type of problem is the number of different ways poker hands can be 
dealt to four players. 

Taken together, the methods described earlier in this chapter and the methods introduced 
in this section form a useful toolbox for solving a wide range of counting problems. When the 
additional methods discussed in Chapter 8 are added to this arsenal, you will be able to solve a 
large percentage of the counting problems that arise in a wide range of areas of study. 


Permutations with Repetition 


Counting permutations when repetition of elements is allowed can easily be done using the 
product rule, as Example | shows. 


How many strings of length r can be formed from the uppercase letters of the English alphabet? 


Solution: By the product rule, because there are 26 uppercase English letters, and because each 
letter can be used repeatedly, we see that there are 26’ strings of uppercase English letters of 
length r. < 


The number of r-permutations of a set with n elements when repetition is allowed is given 
in Theorem 1. 


The number of r-permutations of a set of n objects with repetition allowed is n’. 


Proof: There are n ways to select an element of the set for each of the r positions in the 
r-permutation when repetition is allowed, because for each choice all n objects are available. 
Hence, by the product rule there are n” r-permutations when repetition is allowed. | 
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EXAMPLE 2 


EXAMPLE 3 


Combinations with Repetition 


Consider these examples of combinations with repetition of elements allowed. 


How many ways are there to select four pieces of fruit from a bowl containing apples, oranges, 
and pears if the order in which the pieces are selected does not matter, only the type of fruit and 
not the individual piece matters, and there are at least four pieces of each type of fruit in the 
bowl? 


Solution: To solve this problem we list all the ways possible to select the fruit. There are 15 


ways: 
4 apples 4 oranges 4 pears 
3 apples, | orange 3 apples, 1 pear 3 oranges, | apple 
3 oranges, | pear 3 pears, | apple 3 pears, 1 orange 
2 apples, 2 oranges 2 apples, 2 pears 2 oranges, 2 pears 


2 apples, 1 orange, | pear 2 oranges, | apple, | pear 2 pears, | apple, 1 orange 


The solution is the number of 4-combinations with repetition allowed from a three-element set, 
{apple, orange, pear}. 


To solve more complex counting problems of this type, we need a general method for 
counting the r-combinations of an n-element set. In Example 3 we will illustrate such a method. 


How many ways are there to select five bills from a cash box containing $1 bills, $2 bills, $5 
bills, $10 bills, $20 bills, $50 bills, and $100 bills? Assume that the order in which the bills are 
chosen does not matter, that the bills of each denomination are indistinguishable, and that there 
are at least five bills of each type. 


Solution: Because the order in which the bills are selected does not matter and seven dif- 
ferent types of bills can be selected as many as five times, this problem involves counting 
5-combinations with repetition allowed from a set with seven elements. Listing all possibilities 
would be tedious, because there are a large number of solutions. Instead, we will illustrate the 
use of a technique for counting combinations with repetition allowed. 

Suppose that a cash box has seven compartments, one to hold each type of bill, as illustrated 
in Figure 1. These compartments are separated by six dividers, as shown in the picture. The 
choice of five bills corresponds to placing five markers in the compartments holding different 
types of bills. Figure 2 illustrates this correspondence for three different ways to select five bills, 
where the six dividers are represented by bars and the five bills by stars. 

The number of ways to select five bills corresponds to the number of ways to arrange six 
bars and five stars in a row with a total of 11 positions. Consequently, the number of ways to 
select the five bills is the number of ways to select the positions of the five stars from the 11 
positions. This corresponds to the number of unordered selections of 5 objects from a set of 11 


$100 $50 $20 $10 $5 $2 $1 


FIGURE 1 Cash Box with Seven Types of Bills. 
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FIGURE 2 Examples of Ways to Select Five Bills. 


objects, which can be done in C(11, 5) ways. Consequently, there are 


C(11,5) = ns = 462 
( ’ ) — 516! = 
ways to choose five bills from the cash box with seven types of bills. < 


Theorem 2 generalizes this discussion. 


There are C(n +r —1,r) = C(n+r — 1,n — 1) r-combinations from a set withn elements 
when repetition of elements is allowed. 


Proof: Each r-combination of a set with n elements when repetition is allowed can be rep- 
resented by a list of n — | bars and r stars. The n — | bars are used to mark off n different 
cells, with the ith cell containing a star for each time the ith element of the set occurs in the 
combination. For instance, a 6-combination of a set with four elements is represented with three 
bars and six stars. Here 


wk | «| | ex 


represents the combination containing exactly two of the first element, one of the second element, 
none of the third element, and three of the fourth element of the set. 

As we have seen, each different list containing n — | bars and r stars corresponds to an 
r-combination of the set with n elements, when repetition is allowed. The number of such lists 
is C(n — 1+r,r), because each list corresponds to a choice of the 7 positions to place the r 
stars from the n — | +r positions that contain r stars and n — | bars. The number of such lists 
is also equal to C(n — 1 +17,n — 1), because each list corresponds to a choice of the n — 1 
positions to place the n — | bars. 4 


Examples 4—6 show how Theorem 2 is applied. 


426 6/Counting 


EXAMPLE 4 


Extra p> 
Examples 2 


EXAMPLE 5 


Suppose that a cookie shop has four different kinds of cookies. How many different ways can 
six cookies be chosen? Assume that only the type of cookie, and not the individual cookies or 
the order in which they are chosen, matters. 


Solution: The number of ways to choose six cookies is the number of 6-combinations of a set 
with four elements. From Theorem 2 this equals C(4 + 6 — 1, 6) = C(9, 6). Because 


CO, 6) = C(9,3) = 8 = 84 
’ — | - 1 : 2 : 3 — bi 
there are 84 different ways to choose the six cookies. < 


Theorem 2 can also be used to find the number of solutions of certain linear equations where 
the variables are integers subject to constraints. This is illustrated by Example 5. 


How many solutions does the equation 
xp +2x2+23 = 11 


have, where x1, x2, and x3 are nonnegative integers? 


Solution: To count the number of solutions, we note that a solution corresponds to a way of 
selecting 11 items from a set with three elements so that x; items of type one, x2 items of type 
two, and x3 items of type three are chosen. Hence, the number of solutions is equal to the number 
of 11-combinations with repetition allowed from a set with three elements. From Theorem 2 it 
follows that there are 


13-12 _ 


78 
1-2 


Coa <1, 11) =] 603,11) = C03, 2) = 


solutions. 

The number of solutions of this equation can also be found when the variables are subject 
to constraints. For instance, we can find the number of solutions where the variables are inte- 
gers with x; > 1, x2 > 2, and x3 > 3. A solution to the equation subject to these constraints 
corresponds to a selection of 11 items with x; items of type one, x2 items of type two, and x3 
items of type three, where, in addition, there is at least one item of type one, two items of type 
two, and three items of type three. So, a solution corresponds to a choice of one item of type 
one, two of type two, and three of type three, together with a choice of five additional items of 
any type. By Theorem 2 this can be done in 


7-6 
C(3+5-—1,5)=C(7,5)=C(7,2) = = =21 
ways. Thus, there are 21 solutions of the equation subject to the given constraints. < 


Example 6 shows how counting the number of combinations with repetition allowed arises 
in determining the value of a variable that is incremented each time a certain type of nested loop 
is traversed. 


EXAMPLE 6 


EXAMPLE 7 


Extra po 
Examples 2 
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TABLE 1 Combinations and Permutations With 
and Without Repetition. 


. n! 
r-permutations No ——. 
(n—r)! 
ae n! 
r-combinations No —_——__——. 
r!(n—r)! 
r-permutations Yes n" 
eo (n+r-—1)! 
r-combinations Yes ——__—_ 
r!(n—1)! 


What is the value of k after the following pseudocode has been executed? 


k:=0 
for i; := 1 ton 
for i2 := 1 toi, 


for i, := 1 toin—1 
k:=k+1 


Solution: Note that the initial value of k is 0 and that | is added to k each time the nested loop 
is traversed with a sequence of integers 71, i2, ..., im such that 


l<in Sin-1 S++ SU) S70. 


The number of such sequences of integers is the number of ways to choose m integers from 
{1,2,...,n}, with repetition allowed. (To see this, note that once such a sequence has been 
selected, if we order the integers in the sequence in nondecreasing order, this uniquely defines 
an assignment of im, im—1,.--, 11. Conversely, every such assignment corresponds to a unique 
unordered set.) Hence, from Theorem 2, it follows that k = C(n +m — 1, m) after this code 
has been executed. < 


The formulae for the numbers of ordered and unordered selections of r elements, chosen 
with and without repetition allowed from a set with n elements, are shown in Table 1. 


Permutations with Indistinguishable Objects 


Some elements may be indistinguishable in counting problems. When this is the case, care must 
be taken to avoid counting things more than once. Consider Example 7. 


How many different strings can be made by reordering the letters of the word SUCCESS? 


Solution: Because some of the letters of SUCCESS are the same, the answer is not given by the 
number of permutations of seven letters. This word contains three Ss, two Cs, one U, and one E. 
To determine the number of different strings that can be made by reordering the letters, first note 
that the three Ss can be placed among the seven positions in C(7, 3) different ways, leaving four 
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Links 


positions free. Then the two Cs can be placed in C (4, 2) ways, leaving two free positions. The U 
can be placed in C(2, 1) ways, leaving just one position free. Hence E can be placed in C(1, 1) 
way. Consequently, from the product rule, the number of different strings that can be made is 


Tl! 84! 2! 
CUS CG, 2)Se, eC 314! 212! Id! 110! 
7! 
~ 319NI 1! 
= 490, 


We can prove Theorem 3 using the same sort of reasoning as in Example 7. 


The number of different permutations of n objects, where there are n; indistinguishable 
objects of type 1, m2 indistinguishable objects of type 2, ... , and n, indistinguishable objects 
of type k, is 


n! 


ny!no!---ngl 


Proof: To determine the number of permutations, first note that the n; objects of type one can be 
placed among the n positions in C(n, 11) ways, leaving n — n positions free. Then the objects 
of type two can be placed in C(n — nj, nz) ways, leaving n — n, — n2 positions free. Continue 
placing the objects of type three, ..., type & — 1, until at the last stage, ng objects of type k 
can be placed in C(n — nj — nz —--- — ng—1, Ng) ways. Hence, by the product rule, the total 
number of different permutations is 


C(n,ny)C(n — 11, N2)+++C(n — ny — +++ — Ng-1, Nk) 
_ n! (n — n1)! (n —n, —---—ng_ 1)! 
~ nyt(n—ny)! no! (n— 14 — 12)! nk! O! 
_ n! 
~ nylng!+++ mg! <q 


Distributing Objects into Boxes 


Many counting problems can be solved by enumerating the ways objects can be placed into boxes 
(where the order these objects are placed into the boxes does not matter). The objects can be 
either distinguishable, that is, different from each other, or indistinguishable, that is, considered 
identical. Distinguishable objects are sometimes said to be labeled, whereas indistinguishable 
objects are said to be unlabeled. Similarly, boxes can be distinguishable, that is, different, 
or indinguishable, that is, identical. Distinguishable boxes are often said to be labeled, while 
indistinguishable boxes are said to be unlabeled. When you solve a counting problem using 
the model of distributing objects into boxes, you need to determine whether the objects are 
distinguishable and whether the boxes are distinguishable. Although the context of the counting 
problem makes these two decisions clear, counting problems are sometimes ambiguous and it 
may be unclear which model applies. In such a case it is best to state whatever assumptions you 
are making and explain why the particular model you choose conforms to your assumptions. 


Extra 
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We will see that there are closed formulae for counting the ways to distribute objects, 
distinguishable or indistinguishable, into distinguishable boxes. We are not so lucky when we 
count the ways to distribute objects, distinguishable or indistinguishable, into indistinguishable 
boxes; there are no closed formulae to use in these cases. 


DISTINGUISHABLE OBJECTS AND DISTINGUISHABLE BOXES. We first consider the 
case when distinguishable objects are placed into distinguishable boxes. Consider Example 8 
in which the objects are cards and the boxes are hands of players. 


How many ways are there to distribute hands of 5 cards to each of four players from the standard 
deck of 52 cards? 


Solution: We will use the product rule to solve this problem. To begin, note that the first player 
can be dealt 5 cards in C(52, 5) ways. The second player can be dealt 5 cards in C(47, 5) ways, 
because only 47 cards are left. The third player can be dealt 5 cards in C(42, 5) ways. Finally, 
the fourth player can be dealt 5 cards in C (37, 5) ways. Hence, the total number of ways to deal 
four players 5 cards each is 


C(52, 5)C(47, 5)C (42, 5)C BT, 5) = ee SE OT 
: 7 4751 4215! 3715! 3215! 
52! 


~ 5151515132! | 


Remark: The solution to Example 8 equals the number of permutations of 52 objects, with 5 in- 
distinguishable objects of each of four different types, and 32 objects of a fifth type. This equality 
can be seen by defining a one-to-one correspondence between permutations of this type and dis- 
tributions of cards to the players. To define this correspondence, first order the cards from | to 52. 
Then cards dealt to the first player correspond to the cards in the positions assigned to objects of 
the first type in the permutation. Similarly, cards dealt to the second, third, and fourth players, re- 
spectively, correspond to cards in the positions assigned to objects of the second, third, and fourth 
type, respectively. The cards not dealt to any player correspond to cards in the positions assigned 
to objects of the fifth type. The reader should verify that this is a one-to-one correspondence. 


Example 8 is a typical problem that involves distributing distinguishable objects into dis- 
tinguishable boxes. The distinguishable objects are the 52 cards, and the five distinguishable 
boxes are the hands of the four players and the rest of the deck. Counting problems that involve 
distributing distinguishable objects into boxes can be solved using Theorem 4. 


The number of ways to distribute n distinguishable objects into k distinguishable boxes so 
that n; objects are placed into box 7,7 = 1,2,...,k, equals 


n! 


ni!no!---ng! 


Theorem 4 can be proved using the product rule. We leave the details as Exercise 47. It can also 
be proved (see Exercise 48) by setting up a one-to-one correspondence between the permutations 
counted by Theorem 3 and the ways to distribute objects counted by Theorem 4. 


INDISTINGUISHABLE OBJECTS AND DISTINGUISHABLE BOXES Counting the 
number of ways of placing n indistinguishable objects into k distinguishable boxes turns out to 
be the same as counting the number of n-combinations for a set with k elements when repeti- 
tions are allowed. The reason behind this is that there is a one-to-one correspondence between 
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? 


EXAMPLE 10 


n-combinations from a set with k elements when repetition is allowed and the ways to place 
n indistinguishable balls into k distinguishable boxes. To set up this correspondence, we put a 
ball in the ith bin each time the ith element of the set is included in the n-combination. 


How many ways are there to place 10 indistinguishable balls into eight distinguishable bins? 


Solution: The number of ways to place 10 indistinguishable balls into eight bins equals the num- 
ber of 10-combinations from a set with eight elements when repetition is allowed. Consequently, 
there are 


17! 
10 — 1, 10) = C(17, 10) = —— = 19,448. 
C(8 + 10 — 1, 10) = C(17, 10) TT 9,448 ‘i 


This means that there are C(n +r — 1,n — 1) ways to place r indistinguishable objects 
into n distinguishable boxes. 


DISTINGUISHABLE OBJECTS ANDINDISTINGUISHABLEBOXES Counting the ways 
to place n distinguishable objects into k indistinguishable boxes is more difficult than counting 
the ways to place objects, distinguishable or indistinguishable objects, into distinguishable 
boxes. We illustrate this with an example. 


How many ways are there to put four different employees into three indistinguishable offices, 
when each office can contain any number of employees? 


Solution: We will solve this problem by enumerating all the ways these employees can be placed 
into the offices. We represent the four employees by A, B, C, and D. First, we note that we can 
distribute employees so that all four are put into one office, three are put into one office and 
a fourth is put into a second office, two employees are put into one office and two put into a 
second office, and finally, two are put into one office, and one each put into the other two offices. 
Each way to distribute these employees to these offices can be represented by a way to partition 
the elements A, B, C, and D into disjoint subsets. 

We can put all four employees into one office in exactly one way, represented by 
{{A, B, C, D}}. We can put three employees into one office and the fourth employee into 
a different office in exactly four ways, represented by {{A, B, C}, {D}}, {{A, B, D}, {C}}, 
{{A, C, D}, {B}}, and {{B, C, D}, {A}}. We can put two employees into one office and two 
into a second office in exactly three ways, represented by {{A, B}, {C, D}}, {{A, C}, {B, D}}, 
and {{A, D}, {B, C}}. Finally, we can put two employees into one office, and one each into each 
of the remaining two offices in six ways, represented by {{A, B}, {C}, {D}}, {{A, C}, {B}, {D}}, 
{{A, D}, {B}, (C}}, {{B, Ch, {A}, (DH, ((B, D}}, {A}, {Ch}, and {{C, D}, {A}, {By}. 

Counting all the possibilities, we find that there are 14 ways to put four different employees 
into three indistinguishable offices. Another way to look at this problem is to look at the number 
of offices into which we put employees. Note that there are six ways to put four different 
employees into three indistinguishable offices so that no office is empty, seven ways to put four 
different employees into two indistinguishable offices so that no office is empty, and one way 
to put four employees into one office so that it is not empty. 


There is no simple closed formula for the number of ways to distribute n distinguishable 
objects into j indistinguishable boxes. However, there is a formula involving a summation, 
which we will now describe. Let S(n, j) denote the number of ways to distribute n distinguish- 
able objects into j indistinguishable boxes so that no box is empty. The numbers S(n, j) are 
called Stirling numbers of the second kind. For instance, Example 10 shows that S(4, 3) = 6, 
S(4, 2) = 7, and S(4, 1) = 1. We see that the number of ways to distribute n distinguishable 
objects into k indistinguishable boxes (where the number of boxes that are nonempty equals k, 
k—1,...,2, or 1) equals ae S(n, j). For instance, following the reasoning in Example 10, 
the number of ways to distribute four distinguishable objects into three indistinguishable boxes 


EXAMPLE 11 
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equals S(4, 1) + S(4, 2) + $(4, 3) = 1+7+6= 14. Using the inclusion—exclusion principle 
(see Section 8.6) it can be shown that 


I j-l1 fi 
sin. = FOCI (FG =o" 
* i=0 


Consequently, the number of ways to distribute n distinguishable objects into k indistinguishable 
boxes equals 


k k , i! j 
Ysap= LS ren' (Pa -o 
j=l j=l J" i=0 ’ 


Remark: The reader may be curious about the Stirling numbers of the first kind. A combinatorial 
definition of the signless Stirling numbers of the first kind, the absolute values of the Stirling 
numbers of the first kind, can be found in the preamble to Exercise 47 in the Supplementary 
Exercises. For the definition of Stirling numbers of the first kind, for more information about 
Stirling numbers of the second kind, and to learn more about Stirling numbers of the first kind 
and the relationship between Stirling numbers of the first and second kind, see combinatorics 
textbooks such as [B607], [Br99], and [RoTe05], and Chapter 6 in [MiRo91]. 


INDISTINGUISHABLE OBJECTS AND INDISTINGUISHABLE BOXES Some counting 
problems can be solved by determining the number of ways to distribute indistinguishable objects 
into indistinguishable boxes. We illustrate this principle with an example. 


How many ways are there to pack six copies of the same book into four identical boxes, where 
a box can contain as many as six books? 


Solution: We will enumerate all ways to pack the books. For each way to pack the books, we will 
list the number of books in the box with the largest number of books, followed by the numbers 
of books in each box containing at least one book, in order of decreasing number of books in a 
box. The ways we can pack the books are 


e 


e 


NNFNWRNE 
ee Ronee 
r 


For example, 4, 1, | indicates that one box contains four books, a second box contains a single 
book, and a third box contains a single book (and the fourth box is empty). We conclude that 
there are nine allowable ways to pack the books, because we have listed them all. < 


Observe that distributing n indistinguishable objects into k indistinguishable boxes is the 
same as writing n as the sum of at most k positive integers in nonincreasing order. If aj + a2 + 
----+ aj =n, where a1, a2,..., aj are positive integers with a; > a2 > --- > aj, we say that 
a\,a2,..., da; isa partition of the positive integer n into j positive integers. We see that if px (1) 
is the number of partitions of n into at most k positive integers, then there are pz(n) ways to 
distribute n indistinguishable objects into k indistinguishable boxes. No simple closed formula 
exists for this number. For more information about partitions of positive integers, see [Ro11]. 
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Exercises 


1. 


10. 


11. 


In how many different ways can five elements be selected 
in order from a set with three elements when repetition is 
allowed? 


. Inhow many different ways can five elements be selected 


in order from a set with five elements when repetition is 
allowed? 


How many strings of six letters are there? 


Every day a student randomly chooses a sandwich for 
lunch from a pile of wrapped sandwiches. If there are six 
kinds of sandwiches, how many different ways are there 
for the student to choose sandwiches for the seven days 
of a week if the order in which the sandwiches are chosen 
matters? 


. How many ways are there to assign three jobs to five 


employees if each employee can be given more than one 
job? 

How many ways are there to select five unordered ele- 
ments from a set with three elements when repetition is 
allowed? 


. How many ways are there to select three unordered el- 


ements from a set with five elements when repetition is 
allowed? 


How many different ways are there to choose a dozen 
donuts from the 21 varieties at a donut shop? 


A bagel shop has onion bagels, poppy seed bagels, egg 
bagels, salty bagels, pumpernickel bagels, sesame seed 
bagels, raisin bagels, and plain bagels. How many ways 
are there to choose 

a) six bagels? 

b) a dozen bagels? 

c) two dozen bagels? 

d) a dozen bagels with at least one of each kind? 

e) a dozen bagels with at least three egg bagels and no 
more than two salty bagels? 

A croissant shop has plain croissants, cherry croissants, 

chocolate croissants, almond croissants, apple croissants, 

and broccoli croissants. How many ways are there to 
choose 

a) a dozen croissants? 

b) three dozen croissants? 

c) two dozen croissants with at least two of each kind? 

d) two dozen croissants with no more than two broccoli 
croissants? 

e) two dozen croissants with at least five chocolate crois- 
sants and at least three almond croissants? 

f) two dozen croissants with at least one plain croissant, 
at least two cherry croissants, at least three choco- 
late croissants, at least one almond croissant, at least 
two apple croissants, and no more than three broccoli 
croissants? 

How many ways are there to choose eight coins from a 

piggy bank containing 100 identical pennies and 80 iden- 

tical nickels? 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19 


20. 


21. 


22. 


23. 


24, 


How many different combinations of pennies, nickels, 
dimes, quarters, and half dollars can a piggy bank con- 
tain if it has 20 coins in it? 

A book publisher has 3000 copies of a discrete mathemat- 
ics book. How many ways are there to store these books 
in their three warehouses if the copies of the book are 
indistinguishable? 

How many solutions are there to the equation 


xX, + x2 +%3+%4 = 17, 


where x1, x2, x3, and x4 are nonnegative integers? 
How many solutions are there to the equation 


xp tx. tx34+x4+%5 = 21, 


where x;, i = 1, 2,3, 4,5, is a nonnegative integer such 
that 

a) x, > 1? 

b) x; => 2 fori = 1,2,3,4,5? 

c) O< x; < 10? 

d) O< x, < 3,1 <x <4, and x3 > 15? 

How many solutions are there to the equation 


Xp +xX2+x3+%4+x5 + X6 = 29, 


where x;,i = 1, 2,3, 4, 5, 6,is anonnegative integer such 
that 

a) x; > 1 fori = 1, 2,3,4,5,6? 

b) x) > 1, x2 > 2,.x3 > 3,x4 > 4, x5 > 5, and x6 > 6? 
c) x1 <5? 

d) x1 < 8 and x2 > 8? 

How many strings of 10 ternary digits (0, 1, or 2) are there 
that contain exactly two Os, three 1s, and five 2s? 

How many strings of 20-decimal digits are there that con- 
tain two Os, four Is, three 2s, one 3, two 4s, three 5s, two 
7s, and three 9s? 

Suppose that a large family has 14 children, including two 
sets of identical triplets, three sets of identical twins, and 
two individual children. How many ways are there to seat 
these children in a row of chairs if the identical triplets or 
twins cannot be distinguished from one another? 

How many solutions are there to the inequality 


xy +x2+x3 < 11, 


where x1, x2, and x3 are nonnegative integers? [Hint: In- 
troduce an auxiliary variable x4 such that x; + x2 + .x3+ 
x4 = 11.] 

How many ways are there to distribute six indistinguish- 
able balls into nine distinguishable bins? 

How many ways are there to distribute 12 indistinguish- 
able balls into six distinguishable bins? 

How many ways are there to distribute 12 distinguishable 
objects into six distinguishable boxes so that two objects 
are placed in each box? 

How many ways are there to distribute 15 distinguish- 
able objects into five distinguishable boxes so that the 
boxes have one, two, three, four, and five objects in them, 
respectively. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


37. 


38. 


39. 


40. 


How many positive integers less than 1,000,000 have the 
sum of their digits equal to 19? 


How many positive integers less than 1,000,000 have ex- 
actly one digit equal to 9 and have a sum of digits equal 
to 13? 


There are 10 questions on a discrete mathematics final 
exam. How many ways are there to assign scores to the 
problems if the sum of the scores is 100 and each question 
is worth at least 5 points? 


Show that there are C(n+r—q,|—q2-—-::—4@, 

ln-g-q qr) different unordered selec- 
tions of n objects of r different types that include at 
least q, objects of type one, gz objects of type two,..., 
and q, objects of type r. 


How many different bit strings can be transmitted if the 
string must begin with a 1 bit, must include three ad- 
ditional 1 bits (so that a total of four 1 bits is sent), 
must include a total of 12 0 bits, and must have at least 
two 0 bits following each 1 bit? 


How many different strings can be made from the letters 
in MISSISSIPPI, using all the letters? 


How many different strings can be made from the letters 
in ABRACADABRA, using all the letters? 


How many different strings can be made from the letters 
in AARDVARK, using all the letters, if all three As must 
be consecutive? 


How many different strings can be made from the letters 
in ORONO, using some or all of the letters? 


How many strings with five or more characters can be 
formed from the letters in SEERESS? 


How many strings with seven or more characters can be 
formed from the letters in EVERGREEN? 


How many different bit strings can be formed using 
six 1s and eight 0s? 


A student has three mangos, two papayas, and two kiwi 
fruits. If the student eats one piece of fruit each day, and 
only the type of fruit matters, in how many different ways 
can these fruits be consumed? 


A professor packs her collection of 40 issues of a mathe- 

matics journal in four boxes with 10 issues per box. How 

many ways can she distribute the journals if 

a) each box is numbered, so that they are distinguish- 
able? 

b) the boxes are identical, so that they cannot be distin- 
guished? 

How many ways are there to travel in xyz space from the 

origin (0, 0, 0) to the point (4, 3,5) by taking steps one 

unit in the positive x direction, one unit in the positive y 

direction, or one unit in the positive z direction? (Moving 

in the negative x, y, or z direction is prohibited, so that 

no backtracking is allowed.) 

How many ways are there to travel in xyzw space from 

the origin (0, 0,0, 0) to the point (4, 3,5, 4) by taking 

steps one unit in the positive x, positive y, positive z, or 

positive w direction? 


41. 


42. 


43. 


44. 


45. 


46. 


*47, 


* 48. 


* 49, 
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How many ways are there to deal hands of seven cards to 

each of five players from a standard deck of 52 cards? 

In bridge, the 52 cards of a standard deck are dealt to four 

players. How many different ways are there to deal bridge 

hands to four players? 

How many ways are there to deal hands of five cards to 

each of six players from a deck containing 48 different 

cards? 

In how many ways can a dozen books be placed on four 

distinguishable shelves 

a) if the books are indistinguishable copies of the same 
title? 

b) if no two books are the same, and the positions of the 
books on the shelves matter? [Hint: Break this into 
12 tasks, placing each book separately. Start with the 
sequence 1, 2, 3, 4 to represent the shelves. Repre- 
sent the books by bj, i = 1, 2,..., 12. Place by; to the 
right of one of the terms in 1, 2,3, 4. Then successively 
place b2, b3,..., and bi2.] 

How many ways can n books be placed on k distinguish- 

able shelves 


a) if the books are indistinguishable copies of the same 
title? 

b) if no two books are the same, and the positions of the 
books on the shelves matter? 


A shelf holds 12 books in a row. How many ways are 
there to choose five books so that no two adjacent books 
are chosen? [Hint: Represent the books that are chosen by 
bars and the books not chosen by stars. Count the number 
of sequences of five bars and seven stars so that no two 
bars are adjacent.] 

Use the product rule to prove Theorem 4, by first placing 
objects in the first box, then placing objects in the second 
box, and so on. 

Prove Theorem 4 by first setting up a one-to-one cor- 
respondence between permutations of n objects with n; 
indistinguishable objects of typei,i =1,2,3,...,k, and 
the distributions of n objects in k boxes such that n; ob- 


jects are placed in box i, i = 1,2,3,..., k and then ap- 
plying Theorem 3. 
In this exercise we will prove Theorem 2 by set- 


ting up a one-to-one correspondence between the set 

of r-combinations with repetition allowed of S= 

{1, 2,3,...,m} and the set of r-combinations of the set 

T = {1,2,3,...,.n+r— 1}. 

a) Arrange the elements in an r-combination, with rep- 
etition allowed, of S into an increasing sequence 
xX) < x2 <--- <x,. Show that the sequence formed 
by adding k — 1 to the kth term is strictly increasing. 
Conclude that this sequence is made up of r distinct 


elements from T. 

b) Show that the procedure described in (a) defines 
a one-to-one correspondence between the set of 
r-combinations, with repetition allowed, of S and 
the r-combinations of 7. [Hint: Show the corre- 
spondence can be reversed by associating to the r- 
combination {x,,x2,...,x;} of T, with 1 <x, < 
XQ <+++ <x, <n-+r-—1, the r-combination with 


434 6/ Counting 


50. 


51. 


52. 


53. 


54, 


55. 


56. 


57. 


58. 


59. 


repetition allowed from S$, formed by subtracting 
k — 1 from the kth element.] 

c) Conclude that there are C(n+r—1l,r) r 
combinations with repetition allowed froma set withn 
elements. 

How many ways are there to distribute five distinguish- 

able objects into three indistinguishable boxes? 

How many ways are there to distribute six distinguishable 

objects into four indistinguishable boxes so that each of 

the boxes contains at least one object? 

How many ways are there to put five temporary employ- 

ees into four identical offices? 

How many ways are there to put six temporary employ- 

ees into four identical offices so that there is at least one 

temporary employee in each of these four offices? 

How many ways are there to distribute five indistinguish- 

able objects into three indistinguishable boxes? 

How many ways are there to distribute six indistinguish- 

able objects into four indistinguishable boxes so that each 

of the boxes contains at least one object? 

How many ways are there to pack eight identical DVDs 

into five indistinguishable boxes so that each box contains 

at least one DVD? 

How many ways are there to pack nine identical DVDs 

into three indistinguishable boxes so that each box con- 

tains at least two DVDs? 

How many ways are there to distribute five balls into 

seven boxes if each box must have at most one ball 

in it if 

a) both the balls and boxes are labeled? 

b) the balls are labeled, but the boxes are unlabeled? 

c) the balls are unlabeled, but the boxes are labeled? 

d) both the balls and boxes are unlabeled? 

How many ways are there to distribute five balls into three 

boxes if each box must have at least one ball in it if 

a) both the balls and boxes are labeled? 

b) the balls are labeled, but the boxes are unlabeled? 


60. 


*61. 


62. 


* 63. 


64. 
65. 
66. 


c) the balls are unlabeled, but the boxes are labeled? 
d) both the balls and boxes are unlabeled? 


Suppose that a basketball league has 32 teams, split into 
two conferences of 16 teams each. Each conference is 
split into three divisions. Suppose that the North Central 
Division has five teams. Each of the teams in the North 
Central Division plays four games against each of the 
other teams in this division, three games against each of 
the 11 remaining teams in the conference, and two games 
against each of the 16 teams in the other conference. In 
how many different orders can the games of one of the 
teams in the North Central Division be scheduled? 


Suppose that a weapons inspector must inspect each of 
five different sites twice, visiting one site per day. The 
inspector is free to select the order in which to visit these 
sites, but cannot visit site X, the most suspicious site, on 
two consecutive days. In how many different orders can 
the inspector visit these sites? 


How many different terms are there in the expansion of 
(xj + x2 +-+++X)" after all terms with identical sets 
of exponents are added? 


Prove the Multinomial Theorem: If n is a positive inte- 
ger, then 


(xy t.xg +++ +m)" 


= oi C(n;n1,n2,.. 


ny +ng+e- +p =n 


ny n2 n 
-,Nm)Xy Ka hats 
where 


n! 


C(n; 1, n2,...,Nm) = 


ni! ng!---nNm! 

is a multinomial coefficient. 

Find the expansion of (x + y + z)*. 

Find the coefficient of x3 y2z> in (x + y +.z)!®, 
How many terms are there in the expansion of 


(x + y +2)! 


6.6 Generating Permutations and Combinations 


Introduction 


Methods for counting various types of permutations and combinations were described in the 
previous sections of this chapter, but sometimes permutations or combinations need to be gener- 
ated, not just counted. Consider the following three problems. First, suppose that a salesperson 
must visit six different cities. In which order should these cities be visited to minimize total 
travel time? One way to determine the best order is to determine the travel time for each of the 
6! = 720 different orders in which the cities can be visited and choose the one with the smallest 
travel time. Second, suppose we are given a set of six positive integers and wish to find a subset 
of them that has 100 as their sum, if such a subset exists. One way to find these numbers is to 
generate all 2° = 64 subsets and check the sum of their elements. Third, suppose a laboratory 
has 95 employees. A group of 12 of these employees with a particular set of 25 skills is needed 
for a project. (Each employee can have one or more of these skills.) One way to find such a 


Links 


EXAMPLE 1 
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set of employees is to generate all sets of 12 of these employees and check whether they have 
the desired skills. These examples show that it is often necessary to generate permutations and 
combinations to solve problems. 


Generating Permutations 


Any set with n elements can be placed in one-to-one correspondence with the set {1, 2,3, ..., m}. 
We can list the permutations of any set of n elements by generating the permutations of the n 
smallest positive integers and then replacing these integers with the corresponding elements. 
Many different algorithms have been developed to generate the n! permutations of this set. We 
will describe one of these that is based on the lexicographic (or dictionary) ordering of the set 
of permutations of {1,2,3,...,”}. In this ordering, the permutation a;a2---a, precedes the 
permutation of bj b2 --- by, if for some k, with | < k <n, a, = bj, a2 = bo, ..., ap—, = De-1, 
and ay < b,x. In other words, a permutation of the set of the n smallest positive integers precedes 
(in lexicographic order) a second permutation if the number in this permutation in the first 
position where the two permutations disagree is smaller than the number in that position in the 
second permutation. 


The permutation 23415 of the set {1, 2, 3, 4, 5} precedes the permutation 23514, because these 
permutations agree in the first two positions, but the number in the third position in the first 
permutation, 4, is smaller than the number in the third position in the second permutation, 5. 
Similarly, the permutation 41532 precedes 52143. 


An algorithm for generating the permutations of {1, 2,...,} can be based on a proce- 
dure that constructs the next permutation in lexicographic order following a given permutation 
a1 a2 +--+ dy. We will show how this can be done. First, suppose that a,_| < d,. Interchange a,_| 
and a, to obtain a larger permutation. No other permutation is both larger than the original per- 
mutation and smaller than the permutation obtained by interchanging a,_—, and a,,. For instance, 
the next larger permutation after 234156 is 234165. On the other hand, if a,_; > ay, then a 
larger permutation cannot be obtained by interchanging these last two terms in the permutation. 
Look at the last three integers in the permutation. If a,—-2 < a,—1, then the last three integers in 
the permutation can be rearranged to obtain the next largest permutation. Put the smaller of the 
two integers a,_, and a, that is greater than a,_2 in position n — 2. Then, place the remaining 
integer and a,_2 into the last two positions in increasing order. For instance, the next larger 
permutation after 234165 is 234516. 

On the other hand, if a,—2 > ay—, (and aj—1 > ay), then a larger permutation cannot be 
obtained by permuting the last three terms in the permutation. Based on these observations, a 
general method can be described for producing the next larger permutation in increasing order 
following a given permutation a a2 --- dy. First, find the integers a; and aj41 with aj < aj+1 
and 


Gj41 > G42 >-++ > Gn, 


that is, the last pair of adjacent integers in the permutation where the first integer in the pair is 
smaller than the second. Then, the next larger permutation in lexicographic order is obtained 
by putting in the jth position the least integer among aj+1,q)+2,..., and a, that is greater 
than a; and listing in increasing order the rest of the integers aj, aj+1, ..., @, in positions j + 1 
ton. It is easy to see that there is no other permutation larger than the permutation aa - - - a, but 
smaller than the new permutation produced. (The verification of this fact is left as an exercise 
for the reader.) 


436 6/ Counting 


EXAMPLE 2 
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EXAMPLE 3 


What is the next permutation in lexicographic order after 362541? 


Solution: The last pair of integers aj and aj4, where aj < aj41 is a3 = 2 and aq =5. The 
least integer to the right of 2 that is greater than 2 in the permutation is as = 4. Hence, 4 is 
placed in the third position. Then the integers 2, 5, and | are placed in order in the last three 
positions, giving 125 as the last three positions of the permutation. Hence, the next permutation 
is 364125. < 


To produce the n! permutations of the integers 1, 2,3, ..., n, begin with the smallest permu- 
tation in lexicographic order, namely, 123 ---, and successively apply the procedure described 
for producing the next larger permutation of n! — 1 times. This yields all the permutations of 
the n smallest integers in lexicographic order. 


Generate the permutations of the integers 1, 2, 3 in lexicographic order. 


Solution: Begin with 123. The next permutation is obtained by interchanging 3 and 2 to obtain 
132. Next, because 3 > 2 and 1 < 3, permute the three integers in 132. Put the smaller of 3 
and 2 in the first position, and then put | and 3 in increasing order in positions 2 and 3 to 
obtain 213. This is followed by 231, obtained by interchanging 1| and 3, because | < 3. The next 
larger permutation has 3 in the first position, followed by 1 and 2 in increasing order, namely, 
312. Finally, interchange | and 2 to obtain the last permutation, 321. We have generated the 
permutations of 1, 2, 3 in lexicographic order. They are 123, 132, 213, 231, 312, and 321. < 


Algorithm | displays the procedure for finding the next permutation in lexicographic order 
after a permutation that is notn n—1n—2 ... 2 1, which is the largest permutation. 


ALGORITHM 1 Generating the Next Permutation in Lexicographic Order. 


procedure next permutation(a,a2 ...a,: permutation of 
{1,2,...,n}notequaltonn—1... 2 1) 
j:=n-1 
while aj > aj+1 
j:=j-1 
{j is the largest subscript with aj < q+1} 
k:=n 
while aj > ax 
k:=k-1 
{ax is the smallest integer greater than a; to the right of a;} 
interchange a; and ax 
ri=n 
si=jtl 
while r > s 
interchange a, and as 
ri=r-l1 
si=s4l1 
{this puts the tail end of the permutation after the jth position in increasing order} 
{aj a2 ...dy, 1s now the next permutation } 


Links 


EXAMPLE 4 
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Generating Combinations 


How can we generate all the combinations of the elements of a finite set? Because a combination 
is just a subset, we can use the correspondence between subsets of {a1, a2, ... , d,} and bit strings 
of length n. 

Recall that the bit string corresponding to a subset has a | in position k if ag is in the subset, 
and has a 0 in this position if ag is not in the subset. If all the bit strings of length n can be listed, 
then by the correspondence between subsets and bit strings, a list of all the subsets is obtained. 

Recall that a bit string of length 7 is also the binary expansion of an integer between 0 and 
2” — 1. The 2” bit strings can be listed in order of their increasing size as integers in their binary 
expansions. To produce all binary expansions of length n, start with the bit string 000. . . 00, with 
n zeros. Then, successively find the next expansion until the bit string 111... 11 is obtained. At 
each stage the next binary expansion is found by locating the first position from the right that is 
not a |, then changing all the Is to the right of this position to Os and making this first 0 (from 
the right) a 1. 


Find the next bit string after 10 0010 0111. 
Solution: The first bit from the right that is not a 1 is the fourth bit from the right. Change 


this bit to a 1 and change all the following bits to Os. This produces the next larger bit string, 
10 0010 1000. 


The procedure for producing the next larger bit string after by) by—2...b, bo 1s given as 
Algorithm 2. 


ALGORITHM 2 Generating the Next Larger Bit String. 


procedure next bit string(by;_—1 by—2.. .b, bo: bit string not equal to 11...11) 
i=0 


while b; = 1 
bj =0 
i:=i+1 

al | 


{by—1 bn—2..-b, bo is now the next bit string} 


Next, an algorithm for generating the r-combinations of the set {1,2,3,...,m} will be 
given. An r-combination can be represented by a sequence containing the elements in the 
subset in increasing order. The r-combinations can be listed using lexicographic order on 
these sequences. In this lexicographic ordering, the first r-combination is {1,2,...,7—1,r} 
and the last r-combination is {n —r+1,n—r+2,...,n—1,n}. The next r-combination 
after aja2---a, can be obtained in the following way: First, locate the last element a; in the 
sequence such that a; An —r +i. Then, replace a; with a; + 1 and aj with a; + j —i +1, 
for j =i+1,i+2,...,r. It is left for the reader to show that this produces the next larger 
r-combination in lexicographic order. This procedure is illustrated with Example 5. 


Find the next larger 4-combination of the set {1, 2, 3, 4, 5, 6} after {1, 2, 5, 6}. 


Solution: The last term among the terms a; with a; = 1, a2 = 2, a3 = 5, and a4 = 6 such that 
aj #6 —4+7 is az = 2. To obtain the next larger 4-combination, increment a2 by | to obtain 
a2 = 3. Then set ag = 3+ 1 =4 and a4 =3+2 =5. Hence the next larger 4-combination is 
{1, 3, 4, 5}. < 
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Algorithm 3 displays pseudocode for this procedure. 


ALGORITHM 3 Generating the Next r-Combination in Lexicographic Order. 


procedure next r-combination({a1, a2, ..., a}: proper subset of 
{1,2,...,} not equal to {nm —r+1,...,n} with 


a, <a. <°+::<a,) 
LF 
while a,j =n—r-+i 
i:=i-1 
qj i=aj +1 
for j:=i+ltor 
aj:=aat+j-i 
{{a1,d2,... 


, ay} is now the next combination} 


Exercises 


1. Place these permutations of {1,2,3,4,5} in lexico- 
graphic order: 43521, 15432, 45321, 23451, 23514, 
14532, 21345, 45213, 31452, 31542. 


2. Place these permutations of {1,2,3,4,5,6} in lexico- 
graphic order: 234561, 231456, 165432, 156423, 543216, 
541236, 231465, 314562, 432561, 654321, 654312, 
435612. 


3. The name of a file in a computer directory consists of 
three uppercase letters followed by a digit, where each 
letter is either A, B, or C, and each digit is either 1 or 2. 
List the name of these files in lexicographic order, where 
we order letters using the usual alphabetic order of letters. 


4. Suppose that the name of a file in a computer directory 
consists of three digits followed by two lowercase letters 
and each digit is 0, 1, or 2, and each letter is either a or b. 
List the name of these files in lexicographic order, where 
we order letters using the usual alphabetic order of letters. 


5. Find the next larger permutation in lexicographic order 
after each of these permutations. 
a) 1432 b) 54123 ce) 12453 
d) 45231 e) 6714235 f) 31528764 

6. Find the next larger permutation in lexicographic order 
after each of these permutations. 


a) 1342 b) 45321 c) 13245 


d) 612345 e) 1623547 f) 23587416 


7. Use Algorithm | to generate the 24 permutations of the 
first four positive integers in lexicographic order. 


8. Use Algorithm 2 to list all the subsets of the set {1, 2, 3, 4}. 


9. Use Algorithm 3 to list all the 3-combinations of 
{1, 2, 3, 4, 5}. 


10. Show that Algorithm 1 produces the next larger permu- 
tation in lexicographic order. 


11. Show that Algorithm 3 produces the next larger 
r-combination in lexicographic order after a given 
r-combination. 


12. Develop an algorithm for generating the r-permutations 
of a set of n elements. 
13. List all 3-permutations of {1, 2, 3, 4, 5}. 
The remaining exercises in this section develop another algo- 
rithm for generating the permutations of {1, 2,3, ...,}. This 
algorithm is based on Cantor expansions of integers. Every 
nonnegative integer less than n! has a unique Cantor expan- 
sion 
ail! + a2! +---+a,_{(n— 1)! 
where a; is a nonnegative integer not exceeding i, for i = 


1,2,...,n—1. The integers aj, a2, ..., @,—, are called the 
Cantor digits of this integer. 


Given a permutation of {1,2,...,m}, let a_j,k= 
2,3,...,n, be the number of integers less than k that fol- 
low k in the permutation. For instance, in the permutation 
43215, a, is the number of integers less than 2 that fol- 
low 2, so aj = 1. Similarly, for this example a2 = 2, a3 = 3, 
and a4 = 0. Consider the function from the set of permu- 
tations of {1,2,3,...,} to the set of nonnegative integers 
less than n! that sends a permutation to the integer that has 
a\,q2,..., 4,—1, defined in this way, as its Cantor digits. 
14. Find the Cantor digits a1, a2,...,@,—1 that correspond 

to these permutations. 

a) 246531 b) 12345 ce) 654321 

*15. Show that the correspondence described in the pream- 
ble is a bijection between the set of permutations of 

{1,2,3,...,} and the nonnegative integers less than n!. 


16. Find the permutations of {1, 2,3, 4,5} that correspond 
to these integers with respect to the correspondence be- 
tween Cantor expansions and permutations as described 
in the preamble to Exercise 14. 


Key Terms and Results 


TERMS 


combinatorics: the study of arrangements of objects 

enumeration: the counting of arrangements of objects 

tree diagram: a diagram made up of a root, branches leaving 
the root, and other branches leaving some of the endpoints 
of branches 

permutation: an ordered arrangement of the elements of a set 

r-permutation: an ordered arrangement of r elements of a set 

P(a,r): the number of r-permutations of a set with n elements 

r-combination: an unordered selection of r elements of a set 

C(n,r): the number of r-combinations of a set with n elements 


binomial coefficient : also the number of r-combinations 


of a set with n elements 

combinatorial proof: a proof that uses counting arguments 
rather than algebraic manipulation to prove a result 

Pascal’s triangle: a representation of the binomial coeffi- 
cients where the ith row of the triangle contains (;) for 
PHONG 2 cael 

S(n,j): the Stirling number of the second kind denoting the 
number of ways to distribute n distinguishable objects into 
j indistinguishable boxes so that no box is empty 


RESULTS 


product rule for counting: The number of ways to do a proce- 
dure that consists of two tasks is the product of the number 
of ways to do the first task and the number of ways to do 
the second task after the first task has been done. 

product rule for sets: The number of elements in the Carte- 
sian product of finite sets is the product of the number of 
elements in each set. 

sum rule for counting: The number of ways to do a task in 
one of two ways is the sum of the number of ways to do 
these tasks if they cannot be done simultaneously. 

sum rule for sets: The number of elements in the union of 
pairwise disjoint finite sets is the sum of the numbers of 
elements in these sets. 


Review Questions 


1. Explain how the sum and product rules can be used to 
find the number of bit strings with a length not exceeding 
10. 


2. Explain how to find the number of bit strings of length 
not exceeding 10 that have at least one 0 bit. 
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a) 3 b) 89 ce) 111 

17. Develop an algorithm for producing all permutations of a 
set of n elements based on the correspondence described 
in the preamble to Exercise 14. 


subtraction rule for counting or inclusion-exclusion for 
sets: If a task can be done in either ny ways or n2 ways, 
then the number of ways to do the task is n} + 2 minus 
the number of ways to do the task that are common to the 
two different ways. 

subtraction rule or inclusion—exclusion for sets: The number 
of elements in the union of two sets is the sum of the number 
of elements in these sets minus the number of elements in 
their intersection. 

division rule for counting: There are n/d ways to do a task 
if it can be done using a procedure that can be carried out 
in n ways, and for every way w, exactly d of the n ways 
correspond to way w. 

division rule for sets: Suppose that a finite set A is the union 
of n disjoint subsets each with d elements. Thenn = |A|/d. 

the pigeonhole principle: When more than k objects are 
placed in k boxes, there must be a box containing more 
than one object. 

the generalized pigeonhole principle: When N objects are 
placed in k boxes, there must be a box containing at least 
[N/k] objects. 


n!} 


(n—r)! 


n n! 
mee (") = a<o 


Pascal’s identity: ("7') = (,",) + (7) 

the binomial theorem: (x + y)"” = )-¢_o (j,)x” *y' 

There are n” r-permutations of a set with n elements when 
repetition is allowed. 

There are C(n + r — 1, r) r-combinations of a set with n ele- 
ments when repetition is allowed. 

There are n!/(n,!nz!---nx!) permutations of n objects of k 
types where there are n; indistinguishable objects of type i 
fori = 1,2,3,...,k. 

the algorithm for generating the permutations of the set 
{1,2,...,n} 


P(n,r) = 


3. a) How can the product rule be used to find the number 
of functions from a set with m elements to a set with 
n elements? 


b) How many functions are there from a set with five 
elements to a set with 10 elements? 
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c) How can the product rule be used to find the number 
of one-to-one functions from a set with m elements to 
a set with n elements? 

d) How many one-to-one functions are there from a set 
with five elements to a set with 10 elements? 

e) How many onto functions are there from a set with 
five elements to a set with 10 elements? 


- 


How can you find the number of possible outcomes of a 
playoff between two teams where the first team that wins 
four games wins the playoff? 


vi 


How can you find the number of bit strings of length ten 
that either begin with 101 or end with 010? 


a) State the pigeonhole principle. 

b) Explain how the pigeonhole principle can be used to 
show that among any 11 integers, at least two must 
have the same last digit. 


7. a) State the generalized pigeonhole principle. 
b) Explain how the generalized pigeonhole principle can 
be used to show that among any 91 integers, there are 
at least ten that end with the same digit. 


8. a) What is the difference between an r-combination and 
an r-permutation of a set with n elements? 

b) Derive an equation that relates the number of r-com- 
binations and the number of r-permutations of a set 
with n elements. 

c) How many ways are there to select six students from 
a class of 25 to serve on a committee? 

d) How many ways are there to select six students from 
a class of 25 to hold six different executive positions 
on a committee? 


al 


9. a) What is Pascal’s triangle? 
b) How can a row of Pascal’s triangle be produced from 
the one above it? 


10. What is meant by a combinatorial proof of an identity? 
How is such a proof different from an algebraic one? 


11. Explain how to prove Pascal’s identity using a combina- 
torial argument. 


a) State the binomial theorem. 

b) Explain how to prove the binomial theorem using a 
combinatorial argument. 

c) Find the coefficient of x!y!! in the expansion of 


(2x + 5y)201, 


12. 


Supplementary Exercises 


1. How many ways are there to choose 6 items from 10 dis- 

tinct items when 

a) the items in the choices are ordered and repetition is 
not allowed? 

b) the items in the choices are ordered and repetition is 
allowed? 

c) the items in the choices are unordered and repetition 
is not allowed? 


d) the items in the choices are unordered and repetition 
is allowed? 


13. 


14. 


15. 


16 


. 


17. 


18 


. 


a) Explain how to find a formula for the number of ways 
to select r objects from n objects when repetition is 
allowed and order does not matter. 

b) How many ways are there to select a dozen objects 
from among objects of five different types if objects 
of the same type are indistinguishable? 

c) How many ways are there to select a dozen objects 
from these five different types if there must be at least 
three objects of the first type? 

d) How many ways are there to select a dozen objects 
from these five different types if there cannot be more 
than four objects of the first type? 

e) How many ways are there to select a dozen objects 
from these five different types if there must be at least 
two objects of the first type, but no more than three 
objects of the second type? 


a) Let n and r be positive integers. Explain why the 
number of solutions of the equation x; + x2 +---+ 
Xn =r, where x; is a nonnegative integer for i = 
1,2,3,...,m, equals the number of r-combinations 
of a set with n elements. 

b) How many solutions in nonnegative integers are there 
to the equation x; + x2 + x3 +24 = 17? 

c) How many solutions in positive integers are there to 
the equation in part (b)? 


a) Derive a formula for the number of permutations of n 
objects of k different types, where there are n; indis- 
tinguishable objects of type one, m2 indistinguishable 
objects of type two, ..., and nx indistinguishable ob- 
jects of type k. 

b) How many ways are there to order the letters of the 
word INDISCREETNESS? 


Describe an algorithm for generating all the permutations 
of the set of the n smallest positive integers. 


a) How many ways are there to deal hands of five cards 
to six players from a standard 52-card deck? 

b) How many ways are there to distribute n distinguish- 
able objects into k distinguishable boxes so that n; 
objects are placed in box i? 


Describe an algorithm for generating all the combinations 
of the set of the n smallest positive integers. 


How many ways are there to choose 10 items from 6 dis- 

tinct items when 

a) the items in the choices are ordered and repetition is 
not allowed? 

b) the items in the choices are ordered and repetition is 
allowed? 

c) the items in the choices are unordered and repetition 
is not allowed? 


d) the items in the choices are unordered and repetition 
is allowed? 


10. 


11. 


12. 


. A test contains 100 true/false questions. How many dif- 


ferent ways can a student answer the questions on the test, 
if answers may be left blank? 


. How many strings of length 10 either start with 000 or 


end with 1111? 


. How many bit strings of length 10 over the alphabet 


{a, b, c} have either exactly three as or exactly four bs? 


. The internal telephone numbers in the phone system on a 


campus consist of five digits, with the first digit not equal 
to zero. How many different numbers can be assigned in 
this system? 


. An ice cream parlor has 28 different flavors, 8 different 


kinds of sauce, and 12 toppings. 

a) In how many different ways can a dish of three scoops 
of ice cream be made where each flavor can be used 
more than once and the order of the scoops does not 
matter? 


b 


hu 


How many different kinds of small sundaes are there 
if a small sundae contains one scoop of ice cream, a 
sauce, and a topping? 


c) How many different kinds of large sundaes are there 
if a large sundae contains three scoops of ice cream, 
where each flavor can be used more than once and 
the order of the scoops does not matter; two kinds of 
sauce, where each sauce can be used only once and 
the order of the sauces does not matter; and three top- 
pings, where each topping can be used only once and 
the order of the toppings does not matter? 


. How many positive integers less than 1000 


a) have exactly three decimal digits? 

b) have an odd number of decimal digits? 

c) have at least one decimal digit equal to 9? 

d) have no odd decimal digits? 

e) have two consecutive decimal digits equal to 5? 


f) are palindromes (that is, read the same forward and 
backward)? 


. When the numbers from | to 1000 are written out in deci- 


mal notation, how many of each of these digits are used? 
a) 0 b) 1 c) 2 d) 9 
There are 12 signs of the zodiac. How many people are 


needed to guarantee that at least six of these people have 
the same sign? 


A fortune cookie company makes 213 different fortunes. 
A student eats at a restaurant that uses fortunes from this 
company and gives each customer one fortune cookie at 
the end of a meal. What is the largest possible number 
of times that the student can eat at the restaurant without 
getting the same fortune four times? 


How many people are needed to guarantee that at least 
two were born on the same day of the week and in the 
same month (perhaps in different years)? 


13. 


14. 


15. 


*16. 


*17, 


18. 


19. 


20. 


21. 


22. 


23. 
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Show that given any set of 10 positive integers not ex- 
ceeding 50 there exist at least two different five-element 
subsets of this set that have the same sum. 


A package of baseball cards contains 20 cards. How many 
packages must be purchased to ensure that two cards in 
these packages are identical if there are a total of 550 
different cards? 


a) How many cards must be chosen from a standard deck 
of 52 cards to guarantee that at least two of the four 
aces are chosen? 


b) How many cards must be chosen from a standard deck 
of 52 cards to guarantee that at least two of the four 
aces and at least two of the 13 kinds are chosen? 

c) How many cards must be chosen from a standard deck 
of 52 cards to guarantee that there are at least two cards 
of the same kind? 

d) How many cards must be chosen from a standard deck 
of 52 cards to guarantee that there are at least two cards 
of each of two different kinds? 


Show that in any set of n + 1 positive integers not exceed- 
ing 2n there must be two that are relatively prime. 


Show that in a sequence of m integers there exists one or 
more consecutive terms with a sum divisible by m. 


Show that if five points are picked in the interior of a 
square with a side length of 2, then at least two of these 
points are no farther than /2 apart. 


Show that the decimal expansion of a rational number 
must repeat itself from some point onward. 


Once acomputer worm infects a personal computer via an 
infected e-mail message, it sends a copy of itself to 100 e- 
mail addresses it finds in the electronic message mailbox 
on this personal computer. What is the maximum number 
of different computers this one computer can infect in the 
time it takes for the infected message to be forwarded five 
times? 


How many ways are there to choose a dozen donuts from 
20 varieties 

a) if there are no two donuts of the same variety? 

b) if all donuts are of the same variety? 

c) if there are no restrictions? 


d) if there are at least two varieties among the dozen 
donuts chosen? 


e) if there must be at least six blueberry-filled donuts? 


f) if there can be no more than six blueberry-filled 
donuts? 


Find n if 
a) P(n,2) = 110. 

c) P(n,4) = 12P(n, 2). 
Find n if 

a) C(n,2) = 45. 

c) C(n, 5) = C(n, 2). 


b) P(n,n) = 5040. 


b) C(m, 3) = P(a, 2). 
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24. 


* 25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


* 33. 


34. 


Show that if m and r are nonnegative integers andn > r, 
then 

Pa+i,r)=P(a,r)n+1)/m+1—-r). 
Suppose that S is a set with n elements. How many ordered 
pairs (A, B) are there such that A and B are subsets of S$ 


with A C B? [Hint: Show that each element of S belongs 
to A, B— A,or S— B.] 


Give a combinatorial proof of Corollary 2 of Section 6.4 
by setting up a correspondence between the subsets of a 
set with an even number of elements and the subsets of 
this set with an odd number of elements. [Hint: Take an el- 
ement a in the set. Set up the correspondence by putting a 
in the subset if it is not already in it and taking it out if it 
is in the subset. ] 


Let n and r be integers with 1 < r < n. Show that 
Ctn,r —1)=C(n+ 2,r4+ 1) 
—2C(n+1,r+I)+Ca,r+1). 


Prove using mathematical induction that pad =7C(j,2) = 
C(n + 1, 3) whenever n is an integer greater than 1. 


Show that if n is an integer then 
n 
n 
y 2( ) = 4" 
k=0 i 
Show that 77} 7"_ 
n> 2. 


n—2 n—-1 n n\ + ; 
Show that )7a7 aig Least l= (3) if n is an in- 
teger with n > 3. 


, | = (5) ifn is an integer with 


In this exercise we will derive a formula for the sum of 

the squares of the n smallest positive integers. We will 

count the number of triples (7, 7, k) where i, 7, and k are 

integers such thatO <i <k,O<j<k,andl<k<n 

in two ways. 

a) Show that there are k* such triples with a fixed k. De- 
duce that there are )~y_ k* such triples. 


b) Show that the number of such triples with 
0 <i <j <k and the number of such triples with 
0 <j <i <k both equal C(n + 1, 3). 


c) Show that the number of such triples with 0 <i = 
j <k equals C(n 4+ 1, 2). 


d) Combining part (a) with parts (b) and (c), conclude 
that 
n 
Soe = 2C(n + 1,3) + CM + 1,2) 
k=1 
=n(n+ 1)(2n 4+ 1)/6. 


How many bit strings of length n, where n > 4, contain 
exactly two occurrences of 01? 


Let S be a set. We say that a collection of sub- 
sets Ai, A2,..., An each containing d elements, where 
d> 2, is 2-colorable if it is possible to assign to 
each element of S one of two different colors so that 


* 


35. 


36. 


37. 


38. 


39. 


40. 


41. 


42. 


43. 


44, 


in every subset A; there are elements that have been 

assigned each color. Let m(d) be the largest integer such 

that every collection of fewer than m(d) sets each con- 

taining d elements is 2-colorable. 

a) Show that the collection of all subsets with d elements 
of a set S with 2d — | elements is not 2-colorable. 

b) Show that m(2) = 3. 

*c) Show that m(3) = 7. [Hint: Show that the collec- 

tion {1,3, 5}, {1, 2, 6}, {1,4, 7}, {2, 3,4}, {2,5, 7}, 
{3, 6, 7}, {4, 5, 6} is not 2-colorable. Then show that 
all collections of six sets with three elements each are 
2-colorable.] 

. A professor writes 20 multiple-choice questions, each 
with the possible answer a, b, c, or d, for a discrete 
mathematics test. If the number of questions with a, b, c, 
and d as their answer is 8, 3, 4, and 5, respectively, how 
many different answer keys are possible, if the questions 
can be placed in any order? 


How many different arrangements are there of eight peo- 
ple seated at a round table, where two arrangements are 
considered the same if one can be obtained from the other 
by a rotation? 


How many ways are there to assign 24 students to five 
faculty advisors? 


How many ways are there to choose a dozen apples from a 

bushel containing 20 indistinguishable Delicious apples, 

20 indistinguishable Macintosh apples, and 20 indistin- 

guishable Granny Smith apples, if at least three of each 

kind must be chosen? 

How many solutions are there to the equation xj + x2 + 

x3 = 17, where x1, x2, and x3 are nonnegative integers 

with 

a) xj > 1, x2 > 2, and x3 > 3? 

b) x; < 6 and x3 > 5? 

c) x, <4, x2 < 3, and x3 > 5? 

a) How many different strings can be made from the word 
PEPPERCORN when all the letters are used? 

b) How many of these strings start and end with the 
letter P? 

c) In how many of these strings are the three letter Ps 
consecutive? 

How many subsets of a set with ten elements 

a) have fewer than five elements? 

b) have more than seven elements? 

c) have an odd number of elements? 

A witness to a hit-and-run accident tells the police that 

the license plate of the car in the accident, which contains 

three letters followed by three digits, starts with the let- 

ters AS and contains both the digits 1 and 2. How many 

different license plates can fit this description? 

How many ways are there to put n identical objects into 

m distinct containers so that no container is empty? 

How many ways are there to seat six boys and eight girls 

in a row of chairs so that no two boys are seated next to 

each other? 


45. How many ways are there to distribute six objects to five 
boxes if 
a) both the objects and boxes are labeled? 
b) the objects are labeled, but the boxes are unlabeled? 
c) the objects are unlabeled, but the boxes are labeled? 
d) both the objects and the boxes are unlabeled? 

46. How many ways are there to distribute five objects into 
six boxes if 


a) both the objects and boxes are labeled? 

b) the objects are labeled, but the boxes are unlabeled? 

c) the objects are unlabeled, but the boxes are labeled? 

d) both the objects and the boxes are unlabeled? 
The signless Stirling number of the first kind c(n, k), 
where k and are integers with 1 < k <n, equals the number 
of ways to arrange n people around & circular tables with at 
least one person seated at each table, where two seatings of 
m people around a circular table are considered the same if 
everyone has the same left neighbor and the same right neigh- 
bor. 


47. Find these signless Stirling numbers of the first kind. 


a) c(3,2) b) (4,2) 
c) c(4,3) d) (5,4) 

48. Show that if n is a positive integer, then ee c(n, j) = 
n!. 


49. Show that if n is a positive integer with n > 3, then 
c(n,n — 2) = Bn — 1)C(n, 3)/4. 

*50. Show that if m and k are integers with 1 < k <n, then 
c(n+1,k)=c(n,k —1)+nc(n,k). 

51. Give a combinatorial proof that 2” divides n! whenever n 
is an even positive integer. [Hint: Use Theorem 3 in Sec- 
tion 6.5 to count the number of permutations of 2n objects 
where there are two indistinguishable objects of n differ- 
ent types. 


Computer Projects 
Write programs with these input and output. 


1. Given a positive integer n and a nonnegative integer not 
exceeding n, find the number of r-permutations and r- 
combinations of a set with n elements. 


2. Given positive integers n and r, find the number of 
r-permutations when repetition is allowed and r-com- 
binations when repetition is allowed of a set with n el- 
ements. 


3. Given a sequence of positive integers, find the longest in- 
creasing and the longest decreasing subsequence of the 
sequence. 


* 4, Given an equation x} + x2 +---+ 2%, = C, where C isa 
constant, and x;, x2,..., X, are nonnegative integers, list 
all the solutions. 
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52. How many 11-element RNA sequences consist of 4 As, 
3Cs, 2Us, and 2Gs, and end with CAA? 
Exercises 53 and 54 are based on a discussion in [RoTe09]. 
A method used in the 1960s for sequencing RNA chains used 
enzymes to break chains after certain links. Some enzymes 
break RNA chains after each G link, while others break them 
after each C or U link. Using these enzymes it is sometimes 
possible to correctly sequence all the bases in an RNA chain. 


* 53. Suppose that when an enzyme that breaks RNA chains af- 
ter each G link is applied to a 12-link chain, the fragments 
obtained are G, CCG, AAAG, and UCCG, and when an 
enzyme that breaks RNA chains after each C or U link 
is applied, the fragments obtained are C, C, C, C, GGU, 
and GAAAG. Can you determine the entire 12-link RNA 
chain from these two sets of fragments? If so, what is this 
RNA chain? 


*54, Suppose that when an enzyme that breaks RNA chains af- 
ter each G link is applied to a 12-link chain, the fragments 
obtained are AC, UG, and ACG and when an enzyme that 
breaks RNA chains after each C or U link is applied, 
the fragments obtained are U, GAC, and GAC. Can you 
determine the entire RNA chain from these two sets of 
fragments? If so, what is this RNA chain? 


55. Devise an algorithm for generating all the r-permutations 
of a finite set when repetition is allowed. 


56. Devise an algorithm for generating all the r-combinations 
of a finite set when repetition is allowed. 


*57. Show that if m and n are integers with m > 3 andn > 3, 
then R(m,n) < R(m,n—1)+ R(m—1,n). 

*58. Show that R(3, 4) > 7 by showing that in a group of six 
people, where any two people are friends or enemies, there 
are not necessarily three mutual friends or four mutual en- 
emies. 


5. Given a positive integer n, list all the permutations of the 
set {1,2,3,...,} in lexicographic order. 

6. Given a positive integer n and a nonnegative integer r 
not exceeding n, list all the r-combinations of the set 
{1,2,3,...,} in lexicographic order. 

7. Given a positive integer n and a nonnegative integer r 
not exceeding n, list all the r-permutations of the set 
{1,2,3,...,} in lexicographic order. 

8. Given a positive integer n, list all the combinations of the 
set {1,2,3,...,n}. 

9. Given positive integers n and r, list all the r-permutations, 
with repetition allowed, of the set {1, 2,3, ..., n}. 


10. Given positive integers n andr, list all the r-combinations, 
with repetition allowed, of the set {1, 2,3, ..., nm}. 
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Computations and Explorations 


Use a computational program or programs you have written to do these exercises. 


. Find the number of possible outcomes in a two-team play- 
off when the winner is the first team to win 5 out of 9, 6 
out of 11, 7 out of 13, and 8 out of 15. 


. Which binomial coefficients are odd? Can you formulate a 
conjecture based on numerical evidence? 


. Verify that C(2n, n) is divisible by the square of a prime, 
when n # 1,2, or 4, for as many positive integers n as 
you can. [The theorem that tells that C(2n, n) is divisible 
by the square of a prime with n 4 1, 2, or 4 was proved 
in 1996 by Andrew Granville and Olivier Ramaré. Their 
proof settled a conjecture made in 1980 by Paul Erdés and 
Ron Graham. ] 


Writing Projects 


Respond to these with essays using outside sources. 


. Describe some of the earliest uses of the pigeonhole prin- 
ciple by Dirichlet and other mathematicians. 


. Discuss ways in which the current telephone numbering 
plan can be extended to accommodate the rapid demand 
for more telephone numbers. (See if you can find some 
of the proposals coming from the telecommunications in- 
dustry.) For each new numbering plan you discuss, show 
how to find the number of different telephone numbers it 
supports. 


. Discuss the importance of combinatorial reasoning in gene 
sequencing and related problems involving genomes. 


. Many combinatorial identities are described in this book. 
Find some sources of such identities and describe important 
combinatorial identities besides those already introduced 
in this book. Give some representative proofs, including 
combinatorial ones, of some of these identities. 


. Describe the different models used to model the dis- 
tribution of particles in statistical mechanics, including 


4. 


*5, 


Crmnrnn 


10. 


Find as many odd integers n less than 200 as you can for 
which C(n, |n/2]) is not divisible by the square of a prime. 
Formulate a conjecture based on your evidence. 

For each integer less than 100 determine whether C (2n, n) 
is divisible by 3. Can you formulate a conjecture that tells 
us for which integers n the binomial coefficient C(2n, n) 
is divisible by 3 based on the digits in the base three ex- 
pansion of n? 


. Generate all the permutations of a set with eight elements. 
. Generate all the 6-permutations of a set with nine elements. 
. Generate all combinations of a set with eight elements. 

. Generate all 5-combinations with repetition allowed of a 


set with seven elements. 


Maxwell—Boltzmann, Bose-Einstein, and Fermi—Dirac 
statistics. In each case, describe the counting techniques 
used in the model. 


. Define the Stirling numbers of the first kind and describe 


some of their properties and the identities they satisfy. 


. Describe some of the properties and the identities that Stir- 


ling numbers of the second kind satisfy, including the con- 
nection between Stirling numbers of the first and second 
kinds. 


. Describe the latest discoveries of values and bounds for 


Ramsey numbers. 


. Describe additional ways to generate all the permutations 


of a set with n elements besides those found in Section 6.6. 
Compare these algorithms and the algorithms described 
in the text and exercises of Section 6.6 in terms of their 
computational complexity. 

Describe at least one way to generate all the partitions of 
a positive integer n. (See Exercise 47 in Section 5.3.) 
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Discrete Probability 


C ombinatorics and probability theory share common origins. The theory of probability was 
first developed more than 300 years ago, when certain gambling games were analyzed. 
Although probability theory was originally invented to study gambling, it now plays an essential 
role in a wide variety of disciplines. For example, probability theory is extensively applied in 
the study of genetics, where it can be used to help understand the inheritance of traits. Of course, 
probability still remains an extremely popular part of mathematics because of its applicability 
to gambling, which continues to be an extremely popular human endeavor. 


In computer science, probability theory plays an important role in the study of the com- 
plexity of algorithms. In particular, ideas and techniques from probability theory are used to 
determine the average-case complexity of algorithms. Probabilistic algorithms can be used to 
solve many problems that cannot be easily or practically solved by deterministic algorithms. 
In a probabilistic algorithm, instead of always following the same steps when given the same 
input, as a deterministic algorithm does, the algorithm makes one or more random choices, 
which may lead to different output. In combinatorics, probability theory can even be used to 
show that objects with certain properties exist. The probabilistic method, a technique in com- 
binatorics introduced by Paul Erdés and Alfréd Rényi, shows that an object with a specified 
property exists by showing that there is a positive probability that a randomly constructed object 
has this property. Probability theory can help us answer questions that involve uncertainty, such 
as determining whether we should reject an incoming mail message as spam based on the words 
that appear in the message. 


An Introduction to Discrete Probability 


Introduction 


Probability theory dates back to 1526 when the Italian mathematician, physician, and gambler 
Girolamo Cardano wrote the first known systematic treatment of the subject in his book Liber 
de Ludo Aleae (Book on Games of Chance). (This book was not published until 1663, which 
may have held back the development of probability theory.) In the seventeenth century the 
French mathematician Blaise Pascal determined the odds of winning some popular bets based 
on the outcome when a pair of dice is repeatedly rolled. In the eighteenth century, the French 
mathematician Laplace, who also studied gambling, defined the probability of an event as the 
number of successful outcomes divided by the number of possible outcomes. For instance, the 
probability that a die comes up an odd number when it is rolled is the number of successful 
outcomes—namely, the number of ways it can come up odd—divided by the number of possible 
outcomes—namely, the number of different ways the die can come up. There are a total of 
six possible outcomes—namely, 1, 2,3, 4,5, and 6—and exactly three of these are successful 
outcomes—namely, |, 3, and 5. Hence, the probability that the die comes up an odd number is 
3/6 = 1/2. (Note that it has been assumed that all possible outcomes are equally likely, or, in 
other words, that the die is fair.) 

In this section we will restrict ourselves to experiments that have finitely many, equally likely, 
outcomes. This permits us to use Laplace’s definition of the probability of an event. We will 
continue our study of probability in Section 7.2, where we will study experiments with finitely 
many outcomes that are not necessarily equally likely. In Section 7.2 we will also introduce 
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DEFINITION 1 


The probability of an 
event can never be 


negative or more than 


one! 


EXAMPLE 1 


Extra 
Examples 


EXAMPLE 2 


6 


some key concepts in probability theory, including conditional probability, independence of 
events, and random variables. In Section 7.4 we will introduce the concepts of the expectation 
and variance of a random variable. 


Finite Probability 


An experiment is a procedure that yields one of a given set of possible outcomes. The sample 
space of the experiment is the set of possible outcomes. An event is a subset of the sample 
space. Laplace’s definition of the probability of an event with finitely many possible outcomes 
will now be stated. 


If S is a finite nonempty sample space of equally likely outcomes, and E is an event, that 


E 
is, a subset of S, then the probability of E is p(E) = = 


According to Laplace’s definition, the probability of an event is between 0 and 1. To see this, 
note that if F is an event from a finite sample space S, then 0 < |E| < |S|, because E C S. 
Thus, 0 < p(E) = |E|/|S| < 1. 

Examples 1-7 illustrate how the probability of an event is found. 


An urn contains four blue balls and five red balls. What is the probability that a ball chosen at 
random from the urn is blue? 


Solution: To calculate the probability, note that there are nine possible outcomes, and four of 
these possible outcomes produce a blue ball. Hence, the probability that a blue ball is chosen 


is 4/9. dj 


What is the probability that when two dice are rolled, the sum of the numbers on the two dice 
is 7? 


Solution: There are a total of 36 equally likely possible outcomes when two dice are rolled. 
(The product rule can be used to see this; because each die has six possible outcomes, the total 


GIROLAMO CARDANO (1501-1576) Cardano, born in Pavia, Italy, was the illegitimate child of Fazio 
Cardano, a lawyer, mathematician, and friend of Leonardo da Vinci, and Chiara Micheria, a young widow. 
In spite of illness and poverty, Cardano was able to study at the universities of Pavia and Padua, from where 
he received his medical degree. Cardano was not accepted into Milan’s College of Physicians because of his 
illegitimate birth, as well as his eccentricity and confrontational style. Nevertheless, his medical skills were 
highly regarded. One of his main accomplishments as a physician is the first description of typhoid fever. 
Cardano published more than 100 books on a diverse range of subjects, including medicine, the natural 
sciences, mathematics, gambling, physical inventions and experiments, and astrology. He also wrote a fascinating 
autobiography. In mathematics, Cardano’s book Ars Magna, published in 1545, established the foundations of 


abstract algebra. This was the most comprehensive book on abstract algebra for more than a century; it presents many novel ideas of 
Cardano and of others, including methods for solving cubic and quartic equations from their coefficients. Cardano also made several 
important contributions to cryptography. Cardano was an advocate of education for the deaf, believing, unlike his contemporaries, 
that deaf people could learn to read and write before learning to speak, and could use their minds just as well as hearing people. 

Cardano was often short of money. However, he kept himself solvent through gambling and winning money by beating others 
at chess. His book about games of chance, Liber de Ludo Aleae, written in 1526 (but published in 1663), offers the first systematic 
treatment of probability; it also describes effective ways to cheat. Cardano was considered to be a man of dubious moral character; 
he was often described as a liar, gambler, lecher, and heretic. 


Links 


EXAMPLE 3 


EXAMPLE 4 
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number of outcomes when two dice are rolled is 6 = 36.) There are six successful outcomes, 
namely, (1, 6), (2, 5), (3, 4), (4, 3), (5, 2), and (6, 1), where the values of the first and second 
dice are represented by an ordered pair. Hence, the probability that a seven comes up when two 
fair dice are rolled is 6/36 = 1/6. < 


Lotteries are extremely popular throughout the world. We can easily compute the odds of 
winning different types of lotteries, as illustrated in Examples 3 and 4. (The odd of winning the 
popular Mega Millions and Powerball lotteries are studied in the supplementary exercises.) 


In a lottery, players win a large prize when they pick four digits that match, in the correct order, 
four digits selected by a random mechanical process. A smaller prize is won if only three digits 
are matched. What is the probability that a player wins the large prize? What is the probability 
that a player wins the small prize? 


Solution: There is only one way to choose all four digits correctly. By the product rule, there 
are 10+ = 10,000 ways to choose four digits. Hence, the probability that a player wins the large 
prize is 1/10,000 = 0.0001. 

Players win the smaller prize when they correctly choose exactly three of the four digits. 
Exactly one digit must be wrong to get three digits correct, but not all four correct. By the sum 
tule, to find the number of ways to choose exactly three digits correctly, we add the number of 
ways to choose four digits matching the digits picked in all but the ith position, fori = 1, 2, 3, 4. 

To count the number of successes with the first digit incorrect, note that there are nine 
possible choices for the first digit (all but the one correct digit), and one choice for each of the 
other digits, namely, the correct digits for these slots. Hence, there are nine ways to choose four 
digits where the first digit is incorrect, but the last three are correct. Similarly, there are nine 
ways to choose four digits where the second digit is incorrect, nine with the third digit incorrect, 
and nine with the fourth digit incorrect. Hence, there is a total of 36 ways to choose four digits 
with exactly three of the four digits correct. Thus, the probability that a player wins the smaller 
prize is 36/10,000 = 9/2500 = 0.0036. < 


There are many lotteries now that award enormous prizes to people who correctly choose a set 
of six numbers out of the first n positive integers, where n is usually between 30 and 60. What 
is the probability that a person picks the correct six numbers out of 40? 


Solution: There is only one winning combination. The total number of ways to choose six 
numbers out of 40 is 


40! 


C(40, 6) = —— = 3,838,380. 

\ ) 34! 6! 

Consequently, the probability of picking a winning combination is | /3,838,380 ~ 0.00000026. 
(Here the symbol © means approximately equal to.) < 


PIERRE-SUIMON LAPLACE (1749-1827)  Pierre-Simon Laplace came from humble origins in Normandy. 
In his childhood he was educated in a school run by the Benedictines. At 16 he entered the University of Caen 
intending to study theology. However, he soon realized his true interests were in mathematics. After completing 
his studies, he was named a provisional professor at Caen, and in 1769 he became professor of mathematics at 
the Paris Military School. 

Laplace is best known for his contributions to celestial mechanics, the study of the motions of heavenly bod- 
ies. His Traité de Mécanique Céleste is considered one of the greatest scientific works of the early nineteenth cen- 
tury. Laplace was one of the founders of probability theory and made many contributions to mathematical statis- 
tics. His work in this area is documented in his book Théorie Analytique des Probabilités, in which he defined 


the probability of an event as the ratio of the number of favorable outcomes to the total number of outcomes of an experiment. 
Laplace was famous for his political flexibility. He was loyal, in succession, to the French Republic, Napoleon, and King Louis 
XVIII. This flexibility permitted him to be productive before, during, and after the French Revolution. 
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Links 


EXAMPLE 5 


EXAMPLE 6 


EXAMPLE 7 


Poker, and other card games, are growing in popularity. To win at these games it helps to 
know the probability of different hands. We can find the probability of specific hands that arise 
in card games using the techniques developed so far. A deck of cards contains 52 cards. There 
are 13 different kinds of cards, with four cards of each kind. (Among the terms commonly used 
instead of “kind” are “rank,” “face value,” “denomination,” and “‘value.”’) These kinds are twos, 
threes, fours, fives, sixes, sevens, eights, nines, tens, jacks, queens, kings, and aces. There are 
also four suits: spades, clubs, hearts, and diamonds, each containing 13 cards, with one card of 
each kind in a suit. In many poker games, a hand consists of five cards. 


Find the probability that a hand of five cards in poker contains four cards of one kind. 


Solution: By the product rule, the number of hands of five cards with four cards of one kind 
is the product of the number of ways to pick one kind, the number of ways to pick the four of 
this kind out of the four in the deck of this kind, and the number of ways to pick the fifth card. 
This is 


C(13, C4, C48, 1). 


By Example 11 in Section 6.3 there are C(52, 5) different hands of five cards. Hence, the 
probability that a hand contains four cards of one kind is 


C(13, C4, 4)C48, 1) 13-1- 48 


= = 0.00024. < 
C(52, 5) 2,598,960 


What is the probability that a poker hand contains a full house, that is, three of one kind and 
two of another kind? 


Solution: By the product rule, the number of hands containing a full house is the product of the 
number of ways to pick two kinds in order, the number of ways to pick three out of four for 
the first kind, and the number of ways to pick two out of four for the second kind. (Note that 
the order of the two kinds matters, because, for instance, three queens and two aces is different 
from three aces and two queens.) We see that the number of hands containing a full house is 


P(13, 2)C(4, 3)C(4, 2) = 13-12-4-6 = 3744. 
Because there are C(52, 5) = 2,598,960 poker hands, the probability of a full house is 


3744 

———  ~ 0.0014. < 
2,598,960 

What is the probability that the numbers 11, 4, 17, 39, and 23 are drawn in that order from a bin 

containing 50 balls labeled with the numbers 1, 2, ... , 50 if (a) the ball selected is not returned 

to the bin before the next ball is selected and (b) the ball selected is returned to the bin before 

the next ball is selected? 


Solution: (a) By the product rule, there are 50 - 49 - 48 - 47 - 46 = 254,251,200 ways to select 
the balls because each time a ball is drawn there is one fewer ball to choose from. Consequently, 
the probability that 11, 4, 17, 39, and 23 are drawn in that order is 1/254,251,200. This is an 
example of sampling without replacement. 

(b) By the product rule, there are 50° = 312,500,000 ways to select the balls because there are 
50 possible balls to choose from each time a ball is drawn. Consequently, the probability that 
11, 4, 17, 39, and 23 are drawn in that order is 1/312,500,000. This is an example of sampling 
with replacement. < 


THEOREM 1 


EXAMPLE 8 


THEOREM 2 
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Probabilities of Complements and Unions of Events 


We can use counting techniques to find the probability of events derived from other events. 


Let E be an event in a sample space S. The probability of the event E = S — E, the comple- 
mentary event of E, is given by 


p(E) = 1 — p(B). 


Proof: To find the probability of the event E = S — E, note that |E| = |S| — |E|. Hence, 


oe) = WIN Fly _ ay 
S| S| 


| 


There is an alternative strategy for finding the probability of an event when a direct approach 
does not work well. Instead of determining the probability of the event, the probability of its 
complement can be found. This is often easier to do, as Example 8 shows. 


A sequence of 10 bits is randomly generated. What is the probability that at least one of these 
bits is 0? 


Solution: Let E be the event that at least one of the 10 bits is 0. Then E is the event that all the 
bits are 1s. Because the sample space S is the set of all bit strings of length 10, it follows that 


_ |E| 1 
p(E) p(B) G 510 


4 1 1023 
~ 1024-1024" 


Hence, the probability that the bit string will contain at least one 0 bit is 1023/1024. It is quite 
difficult to find this probability directly without using Theorem 1. < 


We can also find the probability of the union of two events. 


Let E; and E> be events in the sample space S. Then 


p(E1 U £2) = pC) + p(£2) — pF 9 Ed). 


Proof: Using the formula given in Section 2.2 for the number of elements in the union of two 
sets, it follows that 


|E, U Eo| = |Ei| + |£2| — |£1 9 £pI. 
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EXAMPLE 9 


Extra p> 
Examples e@ 


EXAMPLE 10 


Links 


Hence, 
sG0ng= 
[S| 
— |Ei| + [Fol — £19 Bal 
7 [S| 
— |Eil , [E2| [£19 | 
~ {S| "(S| ISI 
= p(E1) + p(E2) — p(E1 0 Ep). q 


What is the probability that a positive integer selected at random from the set of positive integers 
not exceeding 100 is divisible by either 2 or 5? 


Solution: Let E, be the event that the integer selected at random is divisible by 2, and let E2 be 
the event that it is divisible by 5. Then E, U E? is the event that it is divisible by either 2 or 5. 
Also, FE, M E is the event that it is divisible by both 2 and 5, or equivalently, that it is divisible 
by 10. Because |£;| = 50, |£2| = 20, and |E; NM E2| = 10, it follows that 


P(E] U Eo) = p(E\) + p(E2) — p( E19 £2) 


50, 20 10 3 4 
~ 100 100 100 5° 


Probabilistic Reasoning 


A common problem is determining which of two events is more likely. Analyzing the probabil- 
ities of such events can be tricky. Example 10 describes a problem of this type. It discusses a 
famous problem originating with the television game show Let’s Make a Deal and named after 
the host of the show, Monty Hall. 


The Monty Hall Three-Door Puzzle Suppose you are a game show contestant. You have a 
chance to win a large prize. You are asked to select one of three doors to open; the large prize 
is behind one of the three doors and the other two doors are losers. Once you select a door, the 
game show host, who knows what is behind each door, does the following. First, whether or not 
you selected the winning door, he opens one of the other two doors that he knows is a losing 
door (selecting at random if both are losing doors). Then he asks you whether you would like to 
switch doors. Which strategy should you use? Should you change doors or keep your original 
selection, or does it not matter? 


Solution: The probability you select the correct door (before the host opens a door and asks you 
whether you want to change) is 1/3, because the three doors are equally likely to be the correct 
door. The probability this is the correct door does not change once the game show host opens 
one of the other doors, because he will always open a door that the prize is not behind. 

The probability that you selected incorrectly is the probability the prize is behind one of the 
two doors you did not select. Consequently, the probability that you selected incorrectly is 2/3. 
If you selected incorrectly, when the game show host opens a door to show you that the prize is 
not behind it, the prize is behind the other door. You will always win if your initial choice was 
incorrect and you change doors. So, by changing doors, the probability you win is 2/3. In other 
words, you should always change doors when given the chance to do so by the game show host. 
This doubles the probability that you will win. (A more rigorous treatment of this puzzle can be 
found in Exercise 15 of Section 7.3. For much more on this famous puzzle and its variations, 
see [Ro09].) < 


Exercises 


16. 


17. 


18. 


*19, 


20. 


21. 


22. 


. What is the probability that a card selected at random 


from a standard deck of 52 cards is an ace? 


. What is the probability that a fair die comes up six when 


it is rolled? 


. What is the probability that a randomly selected integer 


chosen from the first 100 positive integers is odd? 


. What is the probability that a randomly selected day of a 


leap year (with 366 possible days) is in April? 


. What is the probability that the sum of the numbers on 


two dice is even when they are rolled? 


. What is the probability that a card selected at random 


from a standard deck of 52 cards is an ace or a heart? 


. What is the probability that when a coin is flipped six 


times in a row, it lands heads up every time? 


. What is the probability that a five-card poker hand con- 


tains the ace of hearts? 


. What is the probability that a five-card poker hand does 


not contain the queen of hearts? 


. What is the probability that a five-card poker hand con- 


tains the two of diamonds and the three of spades? 


. What is the probability that a five-card poker hand con- 


tains the two of diamonds, the three of spades, the six of 
hearts, the ten of clubs, and the king of hearts? 


. What is the probability that a five-card poker hand con- 


tains exactly one ace? 


. What is the probability that a five-card poker hand con- 


tains at least one ace? 


. What is the probability that a five-card poker hand con- 


tains cards of five different kinds? 


. What is the probability that a five-card poker hand con- 


tains two pairs (that is, two of each of two different kinds 
and a fifth card of a third kind)? 


What is the probability that a five-card poker hand con- 
tains a flush, that is, five cards of the same suit? 


What is the probability that a five-card poker hand con- 
tains a straight, that is, five cards that have consecutive 
kinds? (Note that an ace can be considered either the low- 
est card of an A-2-3-4-5 straight or the highest card of a 
10-J-Q-K-A straight.) 

What is the probability that a five-card poker hand con- 
tains a straight flush, that is, five cards of the same suit of 
consecutive kinds? 

What is the probability that a five-card poker hand con- 
tains cards of five different kinds and does not contain a 
flush or a straight? 

What is the probability that a five-card poker hand con- 
tains a royal flush, that is, the 10, jack, queen, king, and 
ace of one suit? 

What is the probability that a fair die never comes up an 
even number when it is rolled six times? 


What is the probability that a positive integer not exceed- 
ing 100 selected at random is divisible by 3? 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 
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What is the probability that a positive integer not exceed- 
ing 100 selected at random is divisible by 5 or 7? 

Find the probability of winning a lottery by selecting the 
correct six integers, where the order in which these inte- 
gers are selected does not matter, from the positive inte- 
gers not exceeding 

a) 30. b) 36. c) 42. d) 48. 

Find the probability of winning a lottery by selecting the 
correct six integers, where the order in which these inte- 
gers are selected does not matter, from the positive inte- 
gers not exceeding 

a) 50. b) 52. c) 56. d) 60. 

Find the probability of selecting none of the correct six in- 
tegers in a lottery, where the order in which these integers 
are selected does not matter, from the positive integers not 
exceeding 

a) 40. b) 48. c) 56. d) 64. 

Find the probability of selecting exactly one of the correct 
six integers in a lottery, where the order in which these 
integers are selected does not matter, from the positive 
integers not exceeding 

a) 40. b) 48. c) 56. d) 64. 

In a superlottery, a player selects 7 numbers out of the 
first 80 positive integers. What is the probability that a 
person wins the grand prize by picking 7 numbers that 
are among the 11 numbers selected at random by a com- 
puter. 

In a superlottery, players win a fortune if they choose the 
eight numbers selected by a computer from the positive 
integers not exceeding 100. What is the probability that a 
player wins this superlottery? 

What is the probability that a player of a lottery wins 
the prize offered for correctly choosing five (but not six) 
numbers out of six integers chosen at random from the 
integers between | and 40, inclusive? 

Suppose that 100 people enter a contest and that different 
winners are selected at random for first, second, and third 
prizes. What is the probability that Michelle wins one of 
these prizes if she is one of the contestants? 

Suppose that 100 people enter a contest and that different 
winners are selected at random for first, second, and third 
prizes. What is the probability that Kumar, Janice, and 
Pedro each win a prize if each has entered the contest? 
What is the probability that Abby, Barry, and Sylvia win 
the first, second, and third prizes, respectively, in a draw- 
ing if 200 people enter a contest and 

a) no one can win more than one prize. 

b) winning more than one prize is allowed. 

What is the probability that Bo, Colleen, Jeff, and Rohini 
win the first, second, third, and fourth prizes, respectively, 
in a drawing if 50 people enter a contest and 

a) no one can win more than one prize. 

b) winning more than one prize is allowed. 
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35. In roulette, a wheel with 38 numbers is spun. Of these, 18 


are red, and 18 are black. The other two numbers, which 

are neither black nor red, are 0 and 00. The probability 

that when the wheel is spun it lands on any particular 

number is 1/38. 

a) What is the probability that the wheel lands on a red 
number? 


b) What is the probability that the wheel lands on a black 
number twice in a row? 


c) What is the probability that the wheel lands on 0 or 
00? 

d) What is the probability that in five spins the wheel 
never lands on either 0 or 00? 


e) What is the probability that the wheel lands on one of 
the first six integers on one spin, but does not land on 
any of them on the next spin? 


36. Which is more likely: rolling a total of 8 when two dice 


are rolled or rolling a total of 8 when three dice are rolled? 


37. Which is more likely: rolling a total of 9 when two dice 


are rolled or rolling a total of 9 when three dice are rolled? 


38. Two events FE; and E> are called independent if 


P(E, O E2) = p(E)) p(£2). For each of the following 

pairs of events, which are subsets of the set of all possi- 

ble outcomes when a coin is tossed three times, determine 

whether or not they are independent. 

a) E,: tails comes up with the coin is tossed the first 
time; E2: heads comes up when the coin is tossed the 
second time. 


39. 


40. 


41. 


b) £,: the first coin comes up tails; E2: two, and not 
three, heads come up in a row. 

c) £,: the second coin comes up tails; Fo: two, and not 
three, heads come up in a row. 


(We will study independence of events in more depth in 
Section 7.2.) 


Explain what is wrong with the statement that in the 
Monty Hall Three-Door Puzzle the probability that the 
prize is behind the first door you select and the probabil- 
ity that the prize is behind the other of the two doors that 
Monty does not open are both 1/2, because there are two 
doors left. 


Suppose that instead of three doors, there are four doors 
in the Monty Hall puzzle. What is the probability that you 
win by not changing once the host, who knows what is 
behind each door, opens a losing door and gives you the 
chance to change doors? What is the probability that you 
win by changing the door you select to one of the two 
remaining doors among the three that you did not select? 


This problem was posed by the Chevalier de Méré and 

was solved by Blaise Pascal and Pierre de Fermat. 

a) Find the probability of rolling at least one six when a 
fair die is rolled four times. 

b) Find the probability that a double six comes up at least 
once when a pair of dice is rolled 24 times. Answer the 
query the Chevalier de Méré made to Pascal asking 
whether this probability was greater than 1/2. 

c) Is it more likely that a six comes up at least once when 
a fair die is rolled four times or that a double six comes 
up at least once when a pair of dice is rolled 24 times? 


Probability Theory 


Links 


Introduction 


In Section 7.1 we introduced the notion of the probability of an event. (Recall that an event is a 
subset of the possible outcomes of an experiment.) We defined the probability of an event EF as 
Laplace did, that is, 


p(E) = [eI 
[S| 


the number of outcomes in E divided by the total number of outcomes. This definition assumes 
that all outcomes are equally likely. However, many experiments have outcomes that are not 
equally likely. For instance, a coin may be biased so that it comes up heads twice as often as 
tails. Similarly, the likelihood that the input of a linear search is a particular element in a list, or 
is not in the list, depends on how the input is generated. How can we model the likelihood of 
events in such situations? In this section we will show how to define probabilities of outcomes 
to study probabilities of experiments where outcomes may not be equally likely. 

Suppose that a fair coin is flipped four times, and the first time it comes up heads. Given 
this information, what is the probability that heads comes up three times? To answer this and 


Links 
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similar questions, we will introduce the concept of conditional probability. Does knowing that 
the first flip comes up heads change the probability that heads comes up three times? If not, 
these two events are called independent, a concept studied later in this section. 

Many questions address a particular numerical value associated with the outcome of 
an experiment. For instance, when we flip a coin 100 times, what is the probability that 
exactly 40 heads appear? How many heads should we expect to appear? In this section we 
will introduce random variables, which are functions that associate numerical values to the 
outcomes of experiments. 


Assigning Probabilities 


Let S be the sample space of an experiment with a finite or countable number of outcomes. We 
assign a probability p(s) to each outcome s. We require that two conditions be met: 


(i) O< p(s) <1 foreachs € S$ 


and 


(ii) S> p(s) =1. 


ses 


Condition (7) states that the probability of each outcome is a nonnegative real number no greater 
than 1. Condition (ii) states that the sum of the probabilities of all possible outcomes should 
be 1; that is, when we do the experiment, it is a certainty that one of these outcomes occurs. 
(Note that when the sample space is infinite, )°,.5 p(s) is a convergent infinite series.) This is 
a generalization of Laplace’s definition in which each of n outcomes is assigned a probability 
of 1/n. Indeed, conditions (i) and (ii) are met when Laplace’s definition of probabilities of 
equally likely outcomes is used and S is finite. (See Exercise 4.) 

Note that when there are n possible outcomes, x1, x2, ..., Xn, the two conditions to be met 
are 


(i) O< p(x) <1 fori =1,2,...,n 


and 


@) Do p@=L 


i=1 


The function p from the set of all outcomes of the sample space S is called a probability 
distribution. 

To model an experiment, the probability p(s) assigned to an outcome s should equal the limit 
of the number of times s occurs divided by the number of times the experiment is performed, 
as this number grows without bound. (We will assume that all experiments discussed have 
outcomes that are predictable on the average, so that this limit exists. We also assume that the 
outcomes of successive trials of an experiment do not depend on past results.) 


HISTORICAL NOTE The Chevalier de Méré was a French nobleman, a famous gambler, and a bon vivant. 
He was successful at making bets with odds slightly greater than 1/2 (such as having at least one six come 
up in four tosses of a fair die). His correspondence with Pascal asking about the probability of having at least 
one double six come up when a pair of dice is rolled 24 times led to the development of probability theory. 
According to one account, Pascal wrote to Fermat about the Chevalier saying something like “He’s a good guy 
but, alas, he’s no mathematician.” 
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EXAMPLE 1 


DEFINITION 1 


DEFINITION 2 


EXAMPLE 2 


Remark: We will not discuss probabilities of events when the set of outcomes is not finite or 
countable, such as when the outcome of an experiment can be any real number. In such cases, 
integral calculus is usually required for the study of the probabilities of events. 


We can model experiments in which outcomes are either equally likely or not equally likely 
by choosing the appropriate function p(s), as Example | illustrates. 


What probabilities should we assign to the outcomes H (heads) and T (tails) when a fair coin 


is flipped? What probabilities should be assigned to these outcomes when the coin is biased so 
that heads comes up twice as often as tails? 


Solution: For a fair coin, the probability that heads comes up when the coin is flipped equals 
the probability that tails comes up, so the outcomes are equally likely. Consequently, we assign 


the probability 1/2 to each of the two possible outcomes, that is, p(H) = p(T) = 1/2. 
For the biased coin we have 


p(H) = 2p(T). 
Because 
p(H) + p(T) = 1, 
it follows that 
2p(T) + p(T) = 3p(T) = 1. 


We conclude that p(T) = 1/3 and p(H) = 2/3. < 


Suppose that S is a set with n elements. The uniform distribution assigns the probability 1/n 
to each element of S. 


We now define the probability of an event as the sum of the probabilities of the outcomes 
in this event. 


The probability of the event E is the sum of the probabilities of the outcomes in E. That is, 


P(E) = > pts). 


seE 


(Note that when E is an infinite set, )°,., p(s) is a convergent infinite series.) 


Note that when there are n outcomes in the event E, that is, if E = {a,,a2,..., dy}, then 
p(E) = >*'_, p(ai). Note also that the uniform distribution assigns the same probability to 
an event that Laplace’s original definition of probability assigns to this event. The experiment 
of selecting an element from a sample space with a uniform distribution is called selecting an 
element of S at random. 


Suppose that a die is biased (or loaded) so that 3 appears twice as often as each other number 
but that the other five outcomes are equally likely. What is the probability that an odd number 
appears when we roll this die? 
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Solution: We want to find the probability of the event E = {1, 3, 5}. By Exercise 2, we have 


P(1) = p(2) = p(4) = pS) = p) = 1/7; p@B) = 2/7. 


It follows that 
P(E) = pC) + p(3) + p(5) = 1/7 + 2/7 + 1/7 = 4/7. < 


When possible outcomes are equally likely and there are a finite number of possible out- 
comes, the definition of the probability of an event given in this section (Definition 2) agrees 
with Laplace’s definition (Definition | of Section 7.1). To see this, suppose that there are n 
equally likely outcomes; each possible outcome has probability 1/n, because the sum of their 
probabilities is 1. Suppose the event E contains m outcomes. According to Definition 2, 


m 


1 
i=l 


Because | E| = m and |S| = n, it follows that 


gate 
ee AS 


This is Laplace’s definition of the probability of the event EF. 


Probabilities of Complements and Unions of Events 


The formulae for probabilities of combinations of events in Section 7.1 continue to hold 
when we use Definition 2 to define the probability of an event. For example, Theorem | of 
Section 7.1 asserts that 


p(E) = 1— p(E), 


where E is the complementary event of the event E. This equality also holds when Definition 2 
is used. To see this, note that because the sum of the probabilities of the n possible outcomes 
is 1, and each outcome is either in E or in £, but not in both, we have 


>| p(s) = 1 = p(E) + p@). 


seS 


Hence, p(E) =1-p(E£). 
Under Laplace’s definition, by Theorem 2 in Section 7.1, we have 


P(E) U Ex) = p(£1) + p(E2) — p(E1 9 E2) 


whenever EF) and £2 are events in a sample space S. This also holds when we define the prob- 
ability of an event as we do in this section. To see this, note that p(£; U E2) is the sum of 
the probabilities of the outcomes in E; U £2. When an outcome x is in one, but not both, 
of FE; and E>, p(x) occurs in exactly one of the sums for p(£,) and p(£2). When an 
outcome x is in both EF; and £2, p(x) occurs in the sum for p(£}), in the sum for p(F2), 
and in the sum for p(E; M E2), so it occurs 1 + 1 — 1 = 1 time on the right-hand side. Conse- 
quently, the left-hand side and right-hand side are equal. 
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Also, note that if the events E, and E> are disjoint, then p(E, M E2) = 0, which implies 
that 


P(E, U Ex) = p(E1) + p(E2) — p(E1 9 £2) = p(E1) + p(E2). 


Theorem | generalizes this last formula by providing a formula for the probability of the 
union of pairwise disjoint events. 


THEOREM 1 If Ej, £2, ... is a sequence of pairwise disjoint events in a sample space S, then 


(U9) Eom 


(Note that this theorem applies when the sequence F), £2, ... consists of a finite number or 
a countably infinite number of pairwise disjoint events.) 


We leave the proof of Theorem | to the reader (see Exercises 36 and 37). 


Conditional Probability 


Suppose that we flip a coin three times, and all eight possibilities are equally likely. Moreover, 
Links Giz suppose we know that the event F’, that the first flip comes up tails, occurs. Given this information, 
what is the probability of the event F, that an odd number of tails appears? Because the first 
flip comes up tails, there are only four possible outcomes: TTT, TTH, THT, and THH, where H 
and T represent heads and tails, respectively. An odd number of tails appears only for the 
outcomes TTT and THH. Because the eight outcomes have equal probability, each of the four 
possible outcomes, given that F occurs, should also have an equal probability of 1/4. This 
suggests that we should assign the probability of 2/4 = 1/2 to E, given that F occurs. This 

probability is called the conditional probability of E given F. 
In general, to find the conditional probability of E given F,, we use F as the sample space. 
For an outcome from E to occur, this outcome must also belong to E M F.. With this motivation, 

we make Definition 3. 


DEFINITION 3 Let E and F be events with p(F) > 0. The conditional probability of E given F, denoted 
by p(E | F), is defined as 


p(EN F) 


E|Fy= 
p(E | F) oF) 


EXAMPLE3 _ A bit string of length four is generated at random so that each of the 16 bit strings of length four 
Extra a is equally likely. What is the probability that it contains at least two consecutive Os, given that 
Examples MA its first bit is a 0? (We assume that 0 bits and 1 bits are equally likely.) 


Solution: Let E be the event that a bit string of length four contains at least two consecutive Os, 
and let F’ be the event that the first bit of a bit string of length four is a 0. The probability that a 
bit string of length four has at least two consecutive Os, given that its first bit is a 0, equals 


D(EN F) 


E|F)- 
P(E | F) rF) 


EXAMPLE 4 


Links 


DEFINITION 4 


EXAMPLE 5 
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Because EM F = {0000, 0001, 0010, 0011, 0100}, we see that p(E NM F) = 5/16. Because 
there are eight bit strings of length four that start with a 0, we have p(F) = 8/16 = 1/2. 
Consequently, 
5/16 5 
shine” =. < 
1/2 8 
What is the conditional probability that a family with two children has two boys, given they 
have at least one boy? Assume that each of the possibilities BB, BG, GB, and GG is equally 
likely, where B represents a boy and G represents a girl. (Note that BG represents a family with 
an older boy and a younger girl while GB represents a family with an older girl and a younger 
boy.) 


Solution: Let E be the event that a family with two children has two boys, and let F be 
the event that a family with two children has at least one boy. It follows that EF = {BB}, 
F = {BB, BG, GB}, and EM F = {BB}. Because the four possibilities are equally likely, it 
follows that p(F) = 3/4 and p(E MN F) = 1/4. We conclude that 


pIENF) 1/4 1 <4 


PEIAD=" 4 aja 3 


Independence 


Suppose a coin is flipped three times, as described in the introduction to our discussion of 
conditional probability. Does knowing that the first flip comes up tails (event F’) alter the 
probability that tails comes up an odd number of times (event FE’)? In other words, is it the case 
that p(E | F) = p(E)? This equality is valid for the events E and F, because p(E | F) = 1/2 
and p(E) = 1/2. Because this equality holds, we say that E and F are independent events. 
When two events are independent, the occurrence of one of the events gives no information 
about the probability that the other event occurs. 

Because p(E | F) = p(EN F)/p(F), asking whether p(E | F) = p(£) is the same as 
asking whether p(E  F) = p(E)p(F). This leads to Definition 4. 


The events E and F are independent if and only if p(E N F) = p(E)p(F). 


Suppose E is the event that a randomly generated bit string of length four begins with a 1 
and F is the event that this bit string contains an even number of Is. Are FE and F independent, 
if the 16 bit strings of length four are equally likely? 


Solution: There are eight bit strings of length four that begin with a one: 1000, 1001, 1010, 
1011, 1100, 1101, 1110, and 1111. There are also eight bit strings of length four that contain 
an even number of ones: 0000, 0011, 0101, 0110, 1001, 1010, 1100, 1111. Because there are 
16 bit strings of length four, it follows that 


p(E) = p(F) = 8/16 = 1/2. 

Because E 1 F = {1111, 1100, 1010, 1001}, we see that 
D(EN F) = 4/16 = 1/4. 

Because 
p(E NF) = 1/4 = (1/2)(1/2) = p(E) p(F), 


we conclude that E and F are independent. < 
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EXAMPLE 6 


EXAMPLE 7 


DEFINITION 5 


Links 


Probability has many applications to genetics, as Examples 6 and 7 illustrate. 


Assume, as in Example 4, that each of the four ways a family can have two children is equally 
likely. Are the events E, that a family with two children has two boys, and F, that a family with 
two children has at least one boy, independent? 


Solution: Because E = {BB}, we have p(E) = 1/4. In Example 4 we showed that p(F’) = 3/4 
and that p(EM F) = 1/4. But p(E)p(F) = i ‘T= a Therefore p(E ON F) 4 p(E)p(F), 
so the events E and F are not independent. < 


Ww 


Are the events E, that a family with three children has children of both sexes, and F’, that this 
family has at most one boy, independent? Assume that the eight ways a family can have three 
children are equally likely. 


Solution: By assumption, each of the eight ways a family can have three children, 
BBB, BBG, BGB, BGG, GBB, GBG, GGB, and GGG, has a probability of 1/8. Because 
E ={BBG, BGB, BGG, GBB, GBG, GGB}, F ={BGG, GBG, GGB, GGG}, and 
EN” F = {BGG, GBG, GGB}, it follows that p(E) = 6/8 = 3/4, p(F) = 4/8 = 1/2, and 
p(E 1M F) = 3/8. Because 


3 1 3 
E F)=---=-, 
P(E)p(F) 4°27 3 
it follows that p(E ON F) = p(E)p(F),so E and F are independent. (This conclusion may seem 
surprising. Indeed, if we change the number of children, the conclusion may no longer hold. 
See Exercise 27.) 


PAIRWISE AND MUTUAL INDEPENDENCE We can also define the independence of 
more than two events. However, there are two different types of independence, given in 
Definition 5. 


The events EF}, £2,...,E, are pairwise independent if and only if p(£; 1 Ej) = 
p(E;)p(£;) for all pairs of integers i and j with 1<i<j<n. These events 
are mutually independent if p(E;,N E;,1---N E;,,) = p(Ei,) phi.) --- p(Ei,,) 
whenever i;, j = 1,2,...,m, are integers with 1 < ij <i2 <--- <im <nandm > 2. 


From Definition 5, we see that every set of n mutually independent events is also pairwise 
independent. However, n pairwise independent events are not necessarily mutually independent, 
as we see in Exercise 25 in the Supplementary Exercises. Many theorems about 7 events include 
the hypothesis that these events are mutually independent, and not just pairwise independent. 
We will introduce several such theorems later in this chapter. 


Bernoulli Trials and the Binomial Distribution 


Suppose that an experiment can have only two possible outcomes. For instance, when a bit is 
generated at random, the possible outcomes are 0 and 1. When a coin is flipped, the possible 
outcomes are heads and tails. Each performance of an experiment with two possible outcomes is 
called a Bernoulli trial, after James Bernoulli, who made important contributions to probability 
theory. In general, a possible outcome of a Bernoulli trial is called a success or a failure. If p 
is the probability of a success and gq is the probability of a failure, it follows that p + gq = 1. 

Many problems can be solved by determining the probability of k successes when an ex- 
periment consists of n mutually independent Bernoulli trials. (Bernoulli trials are mutually 
independent if the conditional probability of success on any given trial is p, given any infor- 
mation whatsoever about the outcomes of the other trials.) Consider Example 8. 
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EXAMPLE 8 _ Acoin is biased so that the probability of heads is 2/3. What is the probability that exactly four 


THEOREM 2 


EXAMPLE 9 


Extra > 


Examples 


Links 


heads come up when the coin is flipped seven times, assuming that the flips are independent? 


Solution: There are 2’ = 128 possible outcomes when a coin is flipped seven times. The number 
of ways four of the seven flips can be heads is C(7, 4). Because the seven flips are independent, the 
probability of each of these outcomes (four heads and three tails) is (2/3)4(1/3)>. Consequently, 
the probability that exactly four heads appear is 


soto. 560 
37 2187" 
Following the same reasoning as was used in Example 8, we can find the probability of k 
successes in n independent Bernoulli trials. 


< 


C(7, 4)(2/3)*(1/3)? = 


The probability of exactly k successes in n independent Bernoulli trials, with probability of 
success p and probability of failure g = | — p, is 


C(n, bp ag 


Proof: When n Bernoulli trials are carried out, the outcome is an n-tuple (t,%,..., th), 
where t; = S (for success) or ¢t; = F (for failure) fori = 1,2,...,. Because the n trials are 
independent, the probability of each outcome of n trials consisting of k successes and n — k 
failures (in any order) is p<q”—*. Because there are C(n, k) n-tuples of S’s and F’s that contain 
exactly k S’s, the probability of exactly k successes is 


C(n, k) p*g"*. d 


We denote by b(k;n, p) the probability of k successes in n independent Bernoulli tri- 
als with probability of success p and probability of failure g = 1 — p. Considered as a 
function of k, we call this function the binomial distribution. Theorem 2 tells us that 
b(k; n, p) = C(n, k) pkg". 


Suppose that the probability that a 0 bit is generated is 0.9, that the probability that a 1 bit is 
generated is 0.1, and that bits are generated independently. What is the probability that exactly 
eight 0 bits are generated when 10 bits are generated? 


Solution: By Theorem 2, the probability that exactly eight 0 bits are generated is 


b(8; 10, 0.9) = C(10, 8)(0.9)8(0.1)? = 0.1937102445. < 


JAMES BERNOULLI (1654-1705) James Bernoulli (also known as Jacob I), was born in Basel, Switzerland. 
He is one of the eight prominent mathematicians in the Bernoulli family (see Section 10.1 for the Bernoulli 
family tree of mathematicians). Following his father’s wish, James studied theology and entered the ministry. But 
contrary to the desires of his parents, he also studied mathematics and astronomy. He traveled throughout Europe 
from 1676 to 1682, learning about the latest discoveries in mathematics and the sciences. Upon returning to Basel 
in 1682, he founded a school for mathematics and the sciences. He was appointed professor of mathematics at 
the University of Basel in 1687, remaining in this position for the rest of his life. 

James Bernoulli is best known for the work Ars Conjectandi, published eight years after his death. In 
this work, he described the known results in probability theory and in enumeration, often providing alternative 


proofs of known results. This work also includes the application of probability theory to games of chance and his introduction of the 
theorem known as the law of large numbers. This law states that if € > 0, as n becomes arbitrarily large the probability approaches 1 
that the fraction of times an event E occurs during n trials is within € of p(E). 
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Note that the sum of the probabilities that there are k successes when n independent Bernoulli 
trials are carried out, fork = 0,1, 2,...,n, equals 


n 
> Ca, Dp*g"* = (p +9)" =1, 
k=0 


as should be the case. The first equality in this string of equalities is a consequence of the 
binomial theorem (see Section 6.4). The second equality follows because q = | — p. 


Random Variables 


Many problems are concerned with a numerical value associated with the outcome of an experi- 
ment. For instance, we may be interested in the total number of one bits in a randomly generated 
string of 10 bits; or in the number of times tails come up when a coin is flipped 20 times. To 
study problems of this type we introduce the concept of a random variable. 


A random variable is a function from the sample space of an experiment to the set of real 
numbers. That is, a random variable assigns a real number to each possible outcome. 


Remark: Note that a random variable is a function. It is not a variable, and it is not random! 
The name random variable (the translation of variabile casuale) was introduced by the Italian 
mathematician F. P. Cantelli in 1916. In the late 1940s, the mathematicians, W. Feller and 
J. L. Doob flipped a coin to see whether both would use “random variable” or the more fitting 
term “chance variable.” Feller won; unfortunately “random varible” was used in both books and 
ever since. 


Suppose that a coin is flipped three times. Let X(t) be the random variable that equals the 
number of heads that appear when f is the outcome. Then X (t) takes on the following values: 


X (HHH) = 3, 
X (HHT) = X(HTH) = X(THH) =2, 
X(TTH) = X(THT) = X(HTT) = 1, 
X(TTT) = 0. 


The distribution of a random variable X on a sample space S is the set of pairs (r, p(X =1r)) 
for ally € X(S), where p(X =r) is the probability that X takes the value r. (The set of pairs 
in this distribution is determined by the probabilities p(X =r) forr € X(S).) 


Each of the eight possible outcomes when a fair coin is flipped three times has probability 1/8. 
So, the distribution of the random variable X(t) in Example 10 is determined by the proba- 
bilities P(X = 3) = 1/8, P(X = 2) = 3/8, P(X = 1) = 3/8, and P(X = 0) = 1/8. Conse- 
quently, the distribution of X(t) in Example 10 is the set of pairs (3, 1/8), (2, 3/8), (1, 3/8), 
and (0, 1/8). < 


Let X be the sum of the numbers that appear when a pair of dice is rolled. What are the values 
of this random variable for the 36 possible outcomes (i, 7), where i and j are the numbers that 
appear on the first die and the second die, respectively, when these two dice are rolled? 


EXAMPLE 13 
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Solution: The random variable X takes on the following values: 


X((1, 1) = 2, 

X((1, 2)) = X((2, 1)) =3, 

X((1, 3)) = X((2, 2)) = X(G, 1) = 4, 

X((1, 4)) = X((2, 3)) = X((3, 2)) = X(4, 1) =S, 

X((1,5)) = X((2, 4)) = X(G, 3)) = X(4, 2)) = X(G, 1) = 6, 

X((1, 6)) = X((2, 5)) = X((3, 4) = X(4, 3)) = X(G, 2)) = X((6, 1) = 7, 
X ((2, 6)) = X((3, 5)) = X((4, 4) = XG, 3)) = X (6, 2)) = 8, 

X((3, 6)) = X(4, 5)) = X(5, 4) = X((6, 3)) = 9, 

X ((4, 6)) = X((S, 5)) = X((6, 4)) = 10, 

X(G, 6)) = X((6, 5)) = 11, 

X ((6, 6)) = 12. < 


We will continue our study of random variables in Section 7.4, where we will show how 
they can be used in a variety of applications. 


The Birthday Problem 


A famous puzzle asks for the smallest number of people needed in a room so that it is more likely 
than not that at least two of them have the same day of the year as their birthday. Most people 
find the answer, which we determine in Example 13, to be surprisingly small. After we solve 
this famous problem, we will show how similar reasoning can be adapted to solve a question 
about hashing functions. 


The Birthday Problem What is the minimum number of people who need to be in a room so 
that the probability that at least two of them have the same birthday is greater than 1/2? 


Solution: First, we state some assumptions. We assume that the birthdays of the people in the 
room are independent. Furthermore, we assume that each birthday is equally likely and that 
there are 366 days in the year. (In reality, more people are born on some days of the year than 
others, such as days nine months after some holidays including New Year’s Eve, and only leap 
years have 366 days.) 

To find the probability that at least two of n people in a room have the same birthday, 
we first calculate the probability p, that these people all have different birthdays. Then, the 
probability that at least two people have the same birthday is 1— py. To compute p,, we consider 
the birthdays of the n people in some fixed order. Imagine them entering the room one at a time; 
we will compute the probability that each successive person entering the room has a birthday 
different from those of the people already in the room. 

The birthday of the first person certainly does not match the birthday of someone already in 
the room. The probability that the birthday of the second person is different from that of the first 
person 1s 365/366 because the second person has a different birthday when he or she was born 
on one of the 365 days of the year other than the day the first person was born. (The assumption 
that it is equally likely for someone to be born on any of the 366 days of the year enters into this 
and subsequent steps.) 

The probability that the third person has a birthday different from both the birthdays of 
the first and second people given that these two people have different birthdays is 364/366. In 
general, the probability that the jth person, with 2 < 7 < 366, has a birthday different from the 
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birthdays of the j — 1 people already in the room given that these j — 1 people have different 
birthdays is 
s=—(7— 1). 267 = 7 
366 366 


Because we have assumed that the birthdays of the people in the room are independent, we 
can conclude that the probability that the n people in the room have different birthdays is 


_ 365 364 363. «= 367-1 
~~ 366 366 366 366 


It follows that the probability that among n people there are at least two people with the same 
birthday is 


Pn 


365 364 363 =367—n 
366 366 366 366 


To determine the minimum number of people in the room so that the probability that at 
least two of them have the same birthday is greater than 1/2, we use the formula we have found 
for 1 — p, to compute it for increasing values of n until it becomes greater than 1/2. (There are 
more sophisticated approaches using calculus that can eliminate this computation, but we will 
not use them here.) After considerable computation we find that for n = 22, 1 — py, © 0.475, 
while forn = 23, 1— py © 0.506. Consequently, the minimum number of people needed so that 
the probability that at least two people have the same birthday is greater than 1/2 is 23. < 


1-—pr=1- 


The solution to the birthday problem leads to the solution of the question in Example 14 
about hashing functions. 


Probability of a Collision in Hashing Functions Recall from Section 4.5 that a hashing 
function h(k) is a mapping of the keys (of the records that are to be stored in a database) to 
storage locations. Hashing functions map a large universe of keys (such as the approximately 
300 million Social Security numbers in the United States) to a much smaller set of storage 
locations. A good hashing function yields few collisions, which are mappings of two different 
keys to the same memory location, when relatively few of the records are in play in a given 
application. What is the probability that no two keys are mapped to the same location by a 
hashing function, or, in other words, that there are no collisions? 


Solution: To calculate this probability, we assume that the probability that a randomly selected 
key is mapped to a location is 1/m, where m is the number of available locations, that is, the 
hashing function distributes keys uniformly. (In practice, hashing functions may not satisfy this 
assumption. However, for a good hashing function, this assumption should be close to correct.) 
Furthermore, we assume that the keys of the records selected have an equal probability to be 
any of the elements of the key universe and that these keys are independently selected. 

Suppose that the keys are k,, k2,..., k,. When we add the second record, the probability 
that it is mapped to a location different from the location of the first record, that h(k2) 4 h(k1), 
is (m — 1)/m because there are m — | free locations after the first record has been placed. The 
probability that the third record is mapped to a free location after the first and second records 
have been placed without a collision is (m — 2)/m. In general, the probability that the jth record 
is mapped to a free location after the first 7 — 1 records have been mapped to locations h(k;), 
h(k2),..., h(kj—-1) without collisions is (m — (j — 1))/m because j — | of the m locations are 
taken. 

Because the keys are independent, the probability that all n keys are mapped to different 
locations is 


m—-1m-—2 m—-n+1 


Pa = 
m m m 


Monte Carlo methods 
were invented to help 
develop the first nuclear 
weapons. 
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It follows that the probability that there is at least one collision, that is, at least two keys are 
mapped to the same location, is 


m—-1 m—2 m—n+1 


1-—p,=1- 
m m m 


Techniques from calculus can be used to find the smallest value of n given a value of m such 
that the probability of a collision is greater than a particular threshold. It can be shown that the 
smallest integer n such that the probability of a collision is greater than 1/2 is approximately 
n = 1.177,/m. For example, when m = 1,000,000, the smallest integer n such that the proba- 
bility of a collision is greater than 1/2 is 1178. < 


Monte Carlo Algorithms 


The algorithms discussed so far in this book are all deterministic. That is, each algorithm always 
proceeds in the same way whenever given the same input. However, there are many situations 
where we would like an algorithm to make a random choice at one or more steps. Such a 
situation arises when a deterministic algorithm would have to go through a huge number, or even 
an unknown number, of possible cases. Algorithms that make random choices at one or more 
steps are called probabilistic algorithms. We will discuss a particular class of probabilistic 
algorithms in this section, namely, Monte Carlo algorithms, for decision problems. Monte 
Carlo algorithms always produce answers to problems, but a small probability remains that 
these answers may be incorrect. However, the probability that the answer is incorrect decreases 
rapidly when the algorithm carries out sufficient computation. Decision problems have either 
“true” or “false” as their answer. The designation “Monte Carlo” is a reference to the famous 
casino in Monaco; the use of randomness and the repetitive processes in these algorithms make 
them similar to some gambling games. This name was introduced by the inventors of Monte 
Carlo methods, including Stan Ulam, Enrico Fermi, and John von Neumann. 

A Monte Carlo algorithm for a decision problem uses a sequence of tests. The probability 
that the algorithm answers the decision problem correctly increases as more tests are carried 
out. At each step of the algorithm, possible responses are “true,’ which means that the answer 
is “true” and no additional iterations are needed, or “unknown,” which means that the answer 
could be either “true” or “false.” After running all the iterations in such an algorithm, the final 
answer produced is “true” if at least one iteration yields the answer “true,” and the answer is 
“false” if every iteration yields the answer “unknown.” If the correct answer is “false,” then the 
algorithm answers “false,” because every iteration will yield “unknown.” However, if the correct 
answer is “true,” then the algorithm could answer either “true” or “false,” because it may be 
possible that each iteration produced the response “unknown” even though the correct response 
was “true.” We will show that this possibility becomes extremely unlikely as the number of tests 
increases. 

Suppose that p is the probability that the response of a test is “true,” given that the answer 
is “true.” It follows that 1— p is the probability that the response is “unknown,” given that the 
answer is “true.” Because the algorithm answers “false” when all n iterations yield the answer 
“unknown” and the iterations perform independent tests, the probability of error is (1—p)”. 
When p # 0, this probability approaches 0 as the number of tests increases. Consequently, the 
probability that the algorithm answers “true” when the answer is “true” approaches 1. 


Quality Control (This example is adapted from [AhUI95].) Suppose that a manufacturer 
orders processor chips in batches of size n, where n is a positive integer. The chip maker 
has tested only some of these batches to make sure that all the chips in the batch are good 
(replacing any bad chips found during testing with good ones). In previously untested batches, 
the probability that a particular chip is bad has been observed to be 0.1 when random testing 
is done. The PC manufacturer wants to decide whether all the chips in a batch are good. To 
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A number that passes 
many iterations of a 
probabilistic primality 
test is called an industrial 
strength prime, even 
though it may be 
composite. 


do this, the PC manufacturer can test each chip in a batch to see whether it is good. However, 
this requires n tests. Assuming that each test can be carried out in constant time, these tests 
require O(n) seconds. Can the PC manufacturer determine whether a batch of chips has been 
tested by the chip maker using less time? 


Solution: We can use a Monte Carlo algorithm to determine whether a batch of chips has been 
tested by the chip maker as long as we are willing to accept some probability of error. The 
algorithm is set up to answer the question: “Has this batch of chips not been tested by the 
chip maker?” It proceeds by successively selecting chips at random from the batch and testing 
them one by one. When a bad chip is encountered, the algorithm answers “true” and stops. If 
a tested chip is good, the algorithm answers “unknown” and goes on to the next chip. After 
the algorithm has tested a specified number of chips, say k chips, without getting an answer of 
“true,” the algorithm terminates with the answer “false”; that is, the algorithm concludes that 
the batch is good, that is, that the chip maker has tested all the chips in the batch. 

The only way for this algorithm to answer incorrectly is for it to conclude that an untested 
batch of chips has been tested by the chip maker. The probability that a chip is good, but that 
it came from an untested batch, is | — 0.1 = 0.9. Because the events of testing different chips 
from a batch are independent, the probability that all k steps of the algorithm produce the answer 
“unknown,” given that the batch of chips is untested, is 0.9*. 

By taking k large enough, we can make this probability as small as we like. For example, 
by testing 66 chips, the probability that the algorithm decides a batch has been tested by the 
chip maker is 0.9°°, which is less than 0.001. That is, the probability is less than 1 in 1000 
that the algorithm has answered incorrectly. Note that this probability is independent of n, the 
number of chips in a batch. That is, the Monte Carlo algorithm uses a constant number, or O(1), 
tests and requires O(1) seconds, no matter how many chips are in a batch. As long as the PC 
manufacturer can live with an error rate of less than 1 in 1000, the Monte Carlo algorithm will 
save the PC manufacturer a lot of testing. If a smaller error rate is needed, the PC manufacturer 
can test more chips in each batch; the reader can verify that 132 tests lower the error rate to less 
than | in 1,000,000. < 


Probabilistic Primality Testing In Chapter 4 we remarked that a composite integer, that is, an 
integer greater than one that is not prime, passes Miller’s test (see the preamble to Exercise 44 
in Section 4.4) for fewer than n/4 bases b with | < b <n. This observation is the basis for 
a Monte Carlo algorithm to determine whether an integer greater than one is prime. Because 
large primes play an essential role in public-key cryptography (see Section 4.6), being able to 
generate large primes quickly has become extremely important. 

The goal of the algorithm is to decide the question “Is n composite?” Given an integer n 
greater than one, we select an integer b at random with | < b <n and determine whether n 
passes Miller’s test to the base b. If n fails the test, the answer is “true” because n must be 
composite, and the algorithm ends. Otherwise, we perform the test k times, where k is a positive 
integer. Each time we select a random integer b and determine whether n passes Miller’s test to 
the base b. If the answer is “unknown” at each step, the algorithm answers “false,” that is, it says 
that n is not composite, so that it is prime. The only possibility for the algorithm to return an 
incorrect answer occurs when n is composite, and the answer “unknown” is the output at each 
of the k iterations. The probability that a composite integer n passes Miller’s test for a randomly 
selected base b is less than 1/4. Because the integer b with 1 < b < n is selected at random at 
each iteration and these iterations are independent, the probability that n is composite but the 
algorithm responds that n is prime is less than (1/4)*. By taking k to be sufficiently large, we 
can make this probability extremely small. For example, with 10 iterations, the probability that 
the algorithm decides that n is prime when it really is composite is less than 1 in 1,000,000. 
With 30 iterations, this probability drops to less than | in 10!8, an extremely unlikely event. 

To generate large primes, say with 200 digits, we randomly choose an integer n with 200 
digits and run this algorithm, with 30 iterations. If the algorithm decides that n is prime, we 
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can use it as one of the two primes used in an encryption key for the RSA cryptosystem. If n is 
actually composite and is used as part of the key, the procedures used to decrypt messages will 
not produce the original encrypted message. The key is then discarded and two new possible 
primes are used. 


The Probabilistic Method 


We discussed existence proofs in Chapter | and illustrated the difference between constructive 
existence proofs and nonconstructive existence proofs. The probabilistic method, introduced by 
Paul Erdés and Alfréd Rényi, is a powerful technique that can be used to create nonconstructive 
existence proofs. To use the probabilistic method to prove results about a set S, such as the 
existence of an element in S with a specified property, we assign probabilities to the elements 
of S. We then use methods from probability theory to prove results about the elements of S. 
In particular, we can show that an element with a specified property exists by showing that the 
probability an element x € S has this property is positive. The probabilistic method is based on 
the equivalent statement in Theorem 3. 


THE PROBABILISTIC METHOD If the probability that an element chosen at random 
from a S does not have a particular property is less than 1, there exists an element in S with 
this property. 


An existence proof based on the probabilistic method is nonconstructive because it does not find 
a particular element with the desired property. 

We illustrate the power of the probabilistic method by finding a lower bound for the Ramsey 
number R(k, k). Recall from Section 6.2 that R(k, k) equals the minimum number of people at 
a party needed to ensure that there are at least k mutual friends or & mutual enemies (assuming 
that any two people are friends or enemies). 


If k is an integer with k > 2, then R(k, k) > 2*/?. 


Proof: We note that the theorem holds fork = 2 andk = 3 because R(2, 2) = 2and R(3, 3) = 6, 
as was shown in Section 6.2. Now suppose that k > 4. We will use the probabilistic method to 
show that if there are fewer than 2‘/? people at a party, it is possible that no k of them are mutual 
friends or mutual enemies. This will show that R(k, k) is at least 2/2. 
To use the probabilistic method, we assume that it is equally likely for two people to be 
friends or enemies. (Note that this assumption does not have to be realistic.) Suppose there 
n 


are n people at the party. It follows that there are (7) different sets of k people at this 
party, which we list as $1, So,..., Sir): Let E; be the event that all k people in S; are ei- 


ther mutual friends or mutual enemies. The probability that there are either k mutual friends 


or k mutual enemies among the n people equals p(\);“, Ei). 

According to our assumption it is equally likely for two people to be friends or enemies. 
The probability that two people are friends equals the probability that they are enemies; both 
probabilities equal 1/2. Furthermore, there are ey = k(k—1)/2 pairs of people in S; because 
there are k people in S;. Hence, the probability that all k people in S; are mutual friends and the 
probability that all k people in S; are mutual enemies both equal (1/2)**~/. It follows that 


ple) =20 2). 
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The probability that there are either k mutual friends or k mutual enemies in the group of n 


people equals rn, E;). Using Boole’s inequality (Exercise 15), it follows that 


a) fa) y L\ kad? 
P\ UE =) ocen = (7) -2(5) 


i=l i=l 


By Exercise 17 in Section 6.4, we have (G) < n*/2'-|. Hence, 


k(k-1)/2 k k(k-1)/2 
n\,(1 aie, fe 
Vs 2R-1*\ 9 


Now ifn < Qk/2 we have 


k k(k—1)/2 k(k/2) k(k—1)/2 
a=1(5) 20 2(5) 9) <1, 
ge? at 2 


where the last step follows because k > 4. 


We can now conclude that vn, E;) < 1 when k > 4. Hence, the probability of the 
complementary event, that there is no set of either k mutual friends or mutual enemies at the 
party, is greater than 0. It follows that if n < 2*/?, there is at least one set such that no subset 
of k people are mutual friends or mutual enemies. | 


. What probability should be assigned to the outcome of 
heads when a biased coin is tossed, if heads is three times 
as likely to come up as tails? What probability should be 
assigned to the outcome of tails? 

. Find the probability of each outcome when a loaded die 
is rolled, if a 3 is twice as likely to appear as each of the 
other five numbers on the die. 

. Find the probability of each outcome when a biased die is 
rolled, if rolling a 2 or rolling a 4 is three times as likely 
as rolling each of the other four numbers on the die and 
it is equally likely to roll a 2 or a 4. 

. Show that conditions (7) and (ii) are met under Laplace’s 
definition of probability, when outcomes are equally 
likely. 

. A pair of dice is loaded. The probability that a 4 appears 
on the first die is 2/7, and the probability that a 3 appears 
on the second die is 2/7. Other outcomes for each die 
appear with probability 1/7. What is the probability of 7 
appearing as the sum of the numbers when the two dice 
are rolled? 

. What is the probability of these events when we randomly 
select a permutation of {1, 2, 3}? 

a) 1 precedes 3. 

b) 3 precedes 1. 

c) 3 precedes 1 and 3 precedes 2. 


7. What is the probability of these events when we randomly 


select a permutation of {1, 2, 3, 4}? 


a) 1 precedes 4. 

b) 4 precedes 1. 

c) 4 precedes 1 and 4 precedes 2. 

d) 4 precedes 1, 4 precedes 2, and 4 precedes 3. 
e) 4 precedes 3 and 2 precedes 1. 


8. What is the probability of these events when we randomly 


select a permutation of {1,2,...,} wheren > 4? 


a) 1 precedes 2. 

b) 2 precedes 1. 

c) 1 immediately precedes 2. 

d) n precedes | and n —1 precedes 2. 
e) n precedes | and n precedes 2. 


9. What is the probability of these events when we randomly 


select a permutation of the 26 lowercase letters of the En- 
glish alphabet? 


a) The permutation consists of the letters in reverse al- 
phabetic order. 

b) z is the first letter of the permutation. 

c) z precedes a in the permutation. 

d) a immediately precedes z in the permutation. 

e) a immediately precedes m, which immediately pre- 
cedes z in the permutation. 

f) m,n, and o are in their original places in the permu- 
tation. 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


What is the probability of these events when we randomly 

select a permutation of the 26 lowercase letters of the En- 

glish alphabet? 

a) The first 13 letters of the permutation are in alphabet- 
ical order. 

b) a is the first letter of the permutation and z is the last 
letter. 

c) a and z are next to each other in the permutation. 

d) a and Dd are not next to each other in the permutation. 

e) a and z are separated by at least 23 letters in the per- 
mutation. 

f) z precedes both a and b in the permutation. 

Suppose that E and F are events such that p(E£) = 

0.7 and p(F) = 0.5. Show that p(E U F) > 0.7 and 

D(EN F) > 0.2. 

Suppose that E and F are events such that p(E) = 

0.8 and p(F) = 0.6. Show that p(E U F) > 0.8 and 

D(EN F) => 0.4. 

Show that if E and F are events, then p(EM F) > 

p(E) + p(F) — 1. This is known as Bonferroni’s in- 

equality. 

Use mathematical induction to prove the following gen- 

eralization of Bonferroni’s inequality: 


P(E, E2.0---N En) 
> p(E1) + p(£2)+---+ p(En)—- (a — JV), 
where EF), E2,..., E, aren events. 


Show that if E,, E2,..., E, are events from a finite sam- 
ple space, then 


p(B, U E2U---U E,) 


< p(E)) + p(E2) +--+ + p(En). 


This is known as Boole’s inequality. 

Show that if E and F are independent events, then E 
and F are also independent events. 

If E and F are independent events, prove or disprove 
that F and F are necessarily independent events. 


In Exercises 18, 20, and 21 assume that the year has 366 days 
and all birthdays are equally likely. In Exercise 19 assume it 
is equally likely that a person is born in any given month of 
the year. 


18. 


19. 


a) What is the probability that two people chosen at ran- 
dom were born on the same day of the week? 

b) What is the probability that in a group of n people 
chosen at random, there are at least two born on the 
same day of the week? 

c) How many people chosen at random are needed to 
make the probability greater than 1/2 that there are at 
least two people born on the same day of the week? 

a) What is the probability that two people chosen at ran- 
dom were born during the same month of the year? 

b) What is the probability that in a group of n people 
chosen at random, there are at least two born in the 
same month of the year? 

c) How many people chosen at random are needed to 
make the probability greater than 1/2 that there are at 
least two people born in the same month of the year? 


20. 


21. 


#22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


7.2 Probability Theory 467 


Find the smallest number of people you need to choose 
at random so that the probability that at least one of them 
has a birthday today exceeds 1/2. 


Find the smallest number of people you need to choose 
at random so that the probability that at least two of them 
were both born on April 1 exceeds 1/2. 


February 29 occurs only in leap years. Years divisible 
by 4, but not by 100, are always leap years. Years divisi- 
ble by 100, but not by 400, are not leap years, but years 
divisible by 400 are leap years. 


a) What probability distribution for birthdays should be 
used to reflect how often February 29 occurs? 

b) Using the probability distribution from part (a), what 
is the probability that in a group of n people at least 
two have the same birthday? 


What is the conditional probability that exactly four heads 
appear when a fair coin is flipped five times, given that 
the first flip came up heads? 


What is the conditional probability that exactly four heads 
appear when a fair coin is flipped five times, given that 
the first flip came up tails? 


What is the conditional probability that a randomly gen- 
erated bit string of length four contains at least two con- 
secutive Os, given that the first bit is a 1? (Assume the 
probabilities of a 0 and a | are the same.) 


Let E be the event that a randomly generated bit string 
of length three contains an odd number of 1s, and let F 
be the event that the string starts with 1. Are E and F 
independent? 


Let E and F be the events that a family of n children has 

children of both sexes and has at most one boy, respec- 

tively. Are E and F independent if 

a) n=2? b) n= 4? c) n=5? 

Assume that the probability a child is a boy is 0.51 

and that the sexes of children born into a family are 

independent. What is the probability that a family of five 

children has 

a) exactly three boys? 

b) at least one boy? 

c) at least one girl? 

d) all children of the same sex? 

A group of six people play the game of “odd person out” to 

determine who will buy refreshments. Each person flips 

a fair coin. If there is a person whose outcome is not the 

same as that of any other member of the group, this per- 

son has to buy the refreshments. What is the probability 

that there is an odd person out after the coins are flipped 

once? 

Find the probability that a randomly generated bit string 

of length 10 does not contain a 0 if bits are independent 

and if 

a) a0 bit and a 1 bit are equally likely. 

b) the probability that a bit is a 1 is 0.6. 

c) the probability that the ith bit is a 1 is 1/2! for 
P= 1,2,3,4 00,10. 
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31. 


32. 


33. 


34. 


35. 


36. 


* 37. 


38. 


Find the probability that a family with five children does 

not have a boy, if the sexes of children are independent 

and if 

a) a boy and a girl are equally likely. 

b) the probability of a boy is 0.51. 

c) the probability that the ith child is a boy is 
0.51 — @/100). 

Find the probability that a randomly generated bit string 

of length 10 begins with a 1 or ends with a 00 for the same 

conditions as in parts (a), (b), and (c) of Exercise 30, if 

bits are generated independently. 

Find the probability that the first child of a family with 

five children is a boy or that the last two children of the 


family are girls, for the same conditions as in parts (a), 
(b), and (c) of Exercise 31. 


Find each of the following probabilities when n indepen- 
dent Bernoulli trials are carried out with probability of 
success p. 

a) the probability of no successes 

b) the probability of at least one success 

c) the probability of at most one success 

d) the probability of at least two successes 

Find each of the following probabilities when n indepen- 


dent Bernoulli trials are carried out with probability of 
success p. 


a) the probability of no failures 

b) the probability of at least one failure 

c) the probability of at most one failure 

d) the probability of at least two failures 

Use mathematical induction to prove that if 
E\, E2,..., En is a sequence of n pairwise disjoint 
events in a sample space S, where n is a positive integer, 
then p(Uj1 Ei) = Soy) P(E). 

(Requires calculus) Show that if F;, Fo, ...is an infinite 
sequence of pairwise disjoint events in a sample space S, 
then p(U32, Ei) = D272, p(E;). [Hint: Use Exercise 36 
and take limits.] 

A pair of dice is rolled in a remote location and when you 


ask an honest observer whether at least one die came up 
six, this honest observer answers in the affirmative. 


a) What is the probability that the sum of the numbers 
that came up on the two dice is seven, given the infor- 
mation provided by the honest observer? 


Bayes’ Theorem 


** 39, 


* 40. 


41. 


b) Suppose that the honest observer tells us that at least 
one die came up five. What is the probability the sum 
of the numbers that came up on the dice is seven, given 
this information? 


This exercise employs the probabilistic method to prove a 
result about round-robin tournaments. In a round-robin 
tournament with m players, every two players play one 
game in which one player wins and the other loses. 

We want to find conditions on positive integers m 
and k with k < m such that it is possible for the outcomes 
of the tournament to have the property that for every set 
of k players, there is a player who beats every member 
in this set. So that we can use probabilistic reasoning to 
draw conclusions about round-robin tournaments, we as- 
sume that when two players compete it is equally likely 
that either player wins the game and we assume that the 
outcomes of different games are independent. Let E be 
the event that for every set S with k players, where k is 
a positive integer less than m, there is a player who has 
beaten all & players in S. 


a) Show that p(E) < et p(F;), where Fj is the event 
that there is no player who beats all k players from the 


jth set in a list of the (") sets of k players. 

b) Show that the probability of F; is (1—2-"y"~*. 

c) Conclude from parts (a) and (b) that p(E) < 
‘ale! _ Q-kym—k and, therefore, that there must 
be a tournament with the described property if 
()d-2-h"* <1. 

d) Use part (c) to find values of m such that there is a 
tournament with m players such that for every set S 
of two players, there is a player who has beaten both 
players in S. Repeat for sets of three players. 


Devise a Monte Carlo algorithm that determines whether 
a permutation of the integers 1 through 7 has already been 
sorted (that is, itis in increasing order), or instead, is aran- 
dom permutation. A step of the algorithm should answer 
“true” if it determines the list is not sorted and “unknown” 
otherwise. After k steps, the algorithm decides that the in- 
tegers are sorted if the answer is “unknown” in each step. 
Show that as the number of steps increases, the proba- 
bility that the algorithm produces an incorrect answer is 
extremely small. [Hint: For each step, test whether cer- 
tain elements are in the correct order. Make sure these 
tests are independent.] 


Use pseudocode to write out the probabilistic primality 
test described in Example 16. 


Introduction 


There are many times when we want to assess the probability that a particular event occurs on 
the basis of partial evidence. For example, suppose we know the percentage of people who have 
a particular disease for which there is a very accurate diagnostic test. People who test positive for 
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this disease would like to know the likelihood that they actually have the disease. In this section 
we introduce a result that can be used to determine this probability, namely, the probability that 
a person has the disease given that this person tests positive for it. To use this result, we will 
need to know the percentage of people who do not have the disease but test positive for it and 
the percentage of people who have the disease but test negative for it. 

Similarly, suppose we know the percentage of incoming e-mail messages that are spam. We 
will see that we can determine the likelihood that an incoming e-mail message is spam using 
the occurrence of words in the message. To determine this likelihood, we need to know the 
percentage of incoming messages that are spam, the percentage of spam messages in which 
each of these words occurs, and the percentage of messages that are not spam in which each of 
these words occurs. 

The result that we can use to answer questions such as these is called Bayes’ theorem 
and dates back to the eighteenth century. In the past two decades, Bayes’ theorem has been 
extensively applied to estimate probabilities based on partial evidence in areas as diverse as 
medicine, law, machine learning, engineering, and software development. 


Bayes’ Theorem 


We illustrate the idea behind Bayes’ theorem with an example that shows that when extra 
information is available, we can derive a more realistic estimate that a particular event occurs. 
That is, suppose we know p(F), the probability that an event F occurs, but we have knowledge 
that an event E occurs. Then the conditional probability that F occurs given that EF occurs, 
p(F | E), is a more realistic estimate than p(F’) that F occurs. In Example | we will see that 
we can find p(F | E) when we know p(F), p(E | F), and p(E | F). 


EXAMPLE 1 We have two boxes. The first contains two green balls and seven red balls; the second contains 

four green balls and three red balls. Bob selects a ball by first choosing one of the two boxes at 

Extra <~, random. He then selects one of the balls in this box at random. If Bob has selected a red ball, 
Examples what is the probability that he selected a ball from the first box? 


Solution: Let E be the event that Bob has chosen a red ball; E is the event that Bob has chosen 
a green ball. Let F be the event that Bob has chosen a ball from the first box; F is the event that 
Bob has chosen a ball from the second box. We want to find p(F | E), the probability that the 
ball Bob selected came from the first box, given that it is red. By the definition of conditional 
probability, we have p(F | E) = p(F'M E)/p(£). Can we use the information provided to 
determine both p(F M E) and p(£) so that we can find p(F | FE)? 

First, note that because the first box contains seven red balls out of a total of nine balls, 
we know that p(E | F) = 7/9. Similarly, because the second box contains three red balls 
out of a total of seven balls, we know that p(E | F) = 3/7. We assumed that Bob selects a 
box at random, so p(F) = p(F) = = a Because D(E | F) = p(EN F)/p(F), it follows that 

D(EN F)= p(E | F)p(F) = j = — % [as we remarked earlier, this is one of the quantities 
we need to find to determine p(F | E)]. ie enue Decals P(E | F) = p(EN F)/p(F), it 
follows that p(EN F) = p(E | F)p(F) = 7 = 

We can now find p(£). Note that FE = (E ‘a FC U (EF), where EM F and EN F are 
disjoint sets. (If x belongs to both EN F and EQ F, then x belongs to both F and F, which is 
impossible.) It follows that 


= 4 
E) = p(ENF ENF)= = = rt 
P(E) = pl ) + pC ) is | 14 126 126 126 63 


We have now found both p(F MN E) = 7/18 and p(E) = 38/63. We conclude that 


FOE) 7/18 49 
p(F | E) = 2 Das eas, 
P(E) 38/63 —*76 
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Before we had any extra information, we assumed that the probability that Bob selected the first 
box was 1/2. However, with the extra information that the ball selected at random is red, this 
probability has increased to approximately 0.645. That is, the probability that Bob selected a 
ball from the first box increased from 1/2, when no extra information was available, to 0.645 
once we knew that the ball selected was red. 


Using the same type of reasoning as in Example 1, we can find the conditional probability 
that an event F occurs, given that an event E has occurred, when we know p(E | F), p(E | F), 
and p(F). The result we can obtain is called Bayes’ theorem; it is named after Thomas Bayes, 
an eighteenth-century British mathematician and minister who introduced this result. 


BAYES’ THEOREM — Suppose that £ and F are events from a sample space S such that 
p(E) £0 and p(F) 4 0. Then 


pF |B) = P(E | F)p(F) 
p(E | F)p(F)+ pt | F)p(F) 


Proof: The definition of conditional probability tells us that p(F | E) = p(E F)/p(E) 
and p(E | F) = p(EN F)/p(F). Therefore, p(E OM F) = p(F | E)p(E) and p(EN F) = 
D(E | F)p(F). Equating these two expressions for p(E M F’) shows that 


D(F | E)p(E) = p(E | F)p(F). 
Dividing both sides by p(E), we find that 


P(E | F)p(F) 
F | E) = ——_+~-. 
p(F | E) p(E) 


Next, we show that p(E) = p(E | F)p(F)+ p(E | F)p(F). To see this, first note 
that E=ENS=EN(FUF)=(ENF)U(ENF). Furthermore, ENF and ENF 
are disjoint, because if x € ENF and x €¢ EMF, then x € FA F =. Consequently, 
P(E) = p(EN F)+ p(EMF). We have already shown that p(EO F) = p(E | F)p(F). 
Moreover, we have p(E|F)=p(EMF)/p(F), which shows that p(EN F)= 


p(E | F)p(F). It now follows that 


p(E) = p(EN F)+ p(ENF) = p(E | F)p(F) + p(E | F)p(F). 


To complete the proof we insert this expression for p(E) into the equation p(F | E) = 
D(E | F)p(F)/p(£). We have proved that 


P(E | F)p(F) 
p(E | F)p(F)+ ptE | F)p(F) 


pP(F | E)= 
<j 


APPLYING BAYES’ THEOREM Bayes’ theorem can be used to solve problems that arise 
in many disciplines. Next, we will discuss an application of Bayes’ theorem to medicine. In 
particular, we will illustrate how Bayes’ theorem can be used to assess the probability that 
someone testing positive for a disease actually has this disease. The results obtained from 
Bayes’ theorem are often somewhat surprising, as Example 2 shows. 
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EXAMPLE 2 _ Suppose that one person in 100,000 has a particular rare disease for which there is a fairly 
accurate diagnostic test. This test is correct 99.0% of the time when given to a person selected 
at random who has the disease; it is correct 99.5% of the time when given to a person selected 
at random who does not have the disease. Given this information can we find 


(a) the probability that a person who tests positive for the disease has the disease? 
(b) the probability that a person who tests negative for the disease does not have the disease? 


Should a person who tests positive be very concerned that he or she has the disease? 


Solution: (a) Let F be the event that a person selected at random has the disease, and let E be 
the event that a person selected at random tests positive for the disease. We want to compute 
p(F | E). To use Bayes’ theorem to compute p(F | E) we need to find p(E | F), p(E | F), 
p(F), and p(F). 

We know that one person in 100,000 has this disease, so p(F’) = 1/100,000 = 0.00001 and 
p(F) = 1 — 0.00001 = 0.99999. Because a person who has the disease tests positive 99% of 
the time, we know that p(E | F) = 0.99; this is the probability of a true positive, that a person 
with the disease tests positive. It follows that p(E | F) = 1— p(E | F) = 1—0.99 = 0.01; 
this is the probability of a false negative, that a person who has the disease tests negative. 

Furthermore, because a person who does not have the disease tests negative 99.5% of 
the time, we know that p(E | F) = 0.995. This is the probability of a true negative, that a 
person without the disease tests negative. Finally, we see that p(E | F) = 1 — p(E | F) = 
1 — 0.995 = 0.005; this is the probability of a false positive, that a person without the disease 
tests positive. 

The probability that a person who tests positive for the disease actually has the disease is 
D(F | E). By Bayes’ theorem, we know that 


E | F)p(F 
oP dhe P(E | F)p( )_ _ 
D(E | F)p(F) + pt | F)p(F) 


(0.99) (0.00001) 


= = 0.002. 
(0.99)(0.00001) + (0.005) (0.99999) 


(b) The probability that someone who tests negative for the disease does not have the disease is 
D(F | E). By Bayes’ theorem, we know that 


p(E | F)p(F) 
p(E | F)p(F) + p(E | F)p(F) 


_ (0.995) (0.99999) 
~ (0.995) (0.99999) + (0.01)(0.00001) 


p(F | E)= 


= 0.9999999. 


Consequently, 99.99999% of the people who test negative really do not have the disease. 


In part (a) we showed that only 0.2% of people who test positive for the disease actually 
have the disease. Because the disease is extremely rare, the number of false positives on the 
diagnostic test is far greater than the number of true positives, making the percentage of people 
who test positive who actually have the disease extremely small. People who test positive for 
the diseases should not be overly concerned that they actually have the disease. < 


GENERALIZING BAYES’ THEOREM Note that in the statement of Bayes’ theorem, the 
events F and F are mutually exclusive and cover the entire sample space S (that is, F U F = S). 
We can extend Bayes’ theorem to any collection of mutually exclusive events that cover the entire 
sample space S, in the following way. 
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THEOREM 2 GENERALIZED BAYES’ THEOREM _ Suppose that F is an event from a sample space 
S and that F,, F2,..., F, are mutually exclusive events such that Wey F; = S. Assume that 
pe) Orandip G2) 0 tone — ale 27D hen! 


D(E | Fj) pUR) 
i PE | FE) pth) 


Cee) 


We leave the proof of this generalized version of Bayes’ theorem as Exercise 17. 


Bayesian Spam Filters 


Most electronic mailboxes receive a flood of unwanted and unsolicited messages, known as 
spam. Because spam threatens to overwhelm electronic mail systems, a tremendous amount of 
work has been devoted to filtering it out. Some of the first tools developed for eliminating spam 

were based on Bayes’ theorem, such as Bayesian spam filters. 
A Bayesian spam filter uses information about previously seen e-mail messages to guess 
whether an incoming e-mail message is spam. Bayesian spam filters look for occurrences of 
Links Bix particular words in messages. For a particular word w, the probability that w appears in a spam 
e-mail message is estimated by determining the number of times w appears in a message from 
a large set of messages known to be spam and the number of times it appears in a large set of 


The use of the word spam : : : 
4 messages known not to be spam. When we examine e-mail messages to determine whether they 


for unsolicited e-mail 


comes from a Monty might be spam, we look at words that might be indicators of spam, such as “offer,” “special,” or 
Python comedy sketch “opportunity,” as well as words that might indicate that a message is not spam, such as “mom,” 
about a cafe where the “lunch,” or “Jan” (where Jan is one of your friends). Unfortunately, spam filters sometimes fail 


food product Spam comes 
with everything 
regardless of whether 


to identify a spam message as spam; this is called a false negative. And they sometimes identify 
a message that is not spam as spam; this is called a false positive. When testing for spam, it is 


eustomerswanbit, important to minimize false positives, because filtering out wanted e-mail is much worse than 


Links 


letting some spam through. 


THOMAS BAYES (1702-1761) Thomas Bayes was the son a minister in a religious sect known as the 
Nonconformists. This sect was considered heretical in eighteenth-century Great Britain. Because of the secrecy 
of the Nonconformists, little is known of Thomas Bayes’ life. When Thomas was young, his family moved 
to London. Thomas was likely educated privately; Nonconformist children generally did not attend school. In 
1719 Bayes entered the University of Edinburgh, where he studied logic and theology. He was ordained as a 
Nonconformist minister like his father and began his work as a minister assisting his father. In 1733 he became 
minister of the Presbyterian Chapel in Tunbridge Wells, southeast of London, where he remained minister until 
1752. 

Bayes is best known for his essay on probability published in 1764, three years after his death. This essay 
was sent to the Royal Society by a friend who found it in the papers left behind when Bayes died. In the 
introduction to this essay, Bayes stated that his goal was to find a method that could measure the probability that an event happens, 
assuming that we know nothing about it, but that, under the same circumstances, it has happened a certain proportion of times. 
Bayes’ conclusions were accepted by the great French mathematician Laplace but were later challenged by Boole, who questioned 
them in his book Laws of Thought. Since then Bayes’ techniques have been subject to controversy. 

Bayes also wrote an article that was published posthumously: “An Introduction to the Doctrine of Fluxions, and a Defense 
of the Mathematicians Against the Objections of the Author of The Analyst,’ which supported the logical foundations of calculus. 
Bayes was elected a Fellow of the Royal Society in 1742, with the support of important members of the Society, even though at that 
time he had no published mathematical works. Bayes’ sole known publication during his lifetime was allegedly a mystical book 
entitled Divine Benevolence, discussing the original causation and ultimate purpose of the universe. Although the book is commonly 
attributed to Bayes, no author’s name appeared on the title page, and the entire work is thought to be of dubious provenance. 
Evidence for Bayes’ mathematical talents comes from a notebook that was almost certainly written by Bayes, which contains much 
mathematical work, including discussions of probability, trigonometry, geometry, solutions of equations, series, and differential 
calculus. There are also sections on natural philosophy, in which Bayes looks at topics that include electricity, optics, and celestial 
mechanics. Bayes is also the author of a mathematical publication on asymptotic series, which appeared after his death. 


EXAMPLE 3 
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We will develop some basic Bayesian spam filters. First, suppose we have a set B of messages 
known to be spam and a set G of messages known not to be spam. (For example, users could 
classify messages as spam when they examine them in their inboxes.) We next identify the 
words that occur in B and in G. We count the number of messages in the set containing each 
word to find ng(w) and ng(w), the number of messages containing the word w in the sets B 
and G, respectively. Then, the empirical probability that a spam message contains the word w 
is p(w) =ng(w)/|B|, and the empirical probability that a message that is not spam contains 
the word w is g(w) = nG(w)/|G|. We note that p(w) and g(w) estimate the probabilities that 
an incoming spam message, and an incoming message that is not spam, contain the word w, 
respectively. 

Now suppose we receive a new e-mail message containing the word w. Let S be the event 
that the message is spam. Let E be the event that the message contains the word w. The events S, 
that the message is spam, and S, that the message is not spam, partition the set of all messages. 
Hence, by Bayes’ theorem, the probability that the message is spam, given that it contains the 
word w, is 


E|S)p(S 
p(S| E)= pee | )P(S) _ 
P(E | S)p(S) + ptE | S)p(s) 


To apply this formula, we first estimate p(S), the probability that an incoming message is 
spam, as well as p(S), the probability that the incoming message is not spam. Without prior 
knowledge about the likelihood that an incoming message is spam, for simplicity we assume 
that the message is equally likely to be spam as it is not to be spam. That is, we assume that 
p(S) = p(S) = 1/2. Using this assumption, we find that the probability that a message is spam, 
given that it contains the word w, is 


rE | 8) 
(S| E) = —., 
POE) = ELS) + pS) 


(Note that if we have some empirical data about the ratio of spam messages to messages that are 
not spam, we can change this assumption to produce a better estimate for p(S) and for p(S); 
see Exercise 22.) 

Next, we estimate p(E | S), the conditional probability that the message contains the 
word w given that the message is spam, by p(w). Similarly, we estimate p(E | S), the con- 
ditional probability that the message contains the word w, given that the message is not spam, 
by q(w). Inserting these estimates for p(E | S) and p(E | S) tells us that p(S | E) can be 
estimated by 


pw) 
r(w) = ————_; 
pw) + gw) 
that is, r(w) estimates the probability that the message is spam, given that it contains the 
word w. If r(w) is greater than a threshold that we set, such as 0.9, then we classify the message 
as spam. 


Suppose that we have found that the word “Rolex” occurs in 250 of 2000 messages known to 
be spam and in 5 of 1000 messages known not to be spam. Estimate the probability that an 
incoming message containing the word “Rolex” is spam, assuming that it is equally likely that 
an incoming message is spam or not spam. If our threshold for rejecting a message as spam 
is 0.9, will we reject such messages? 


Solution: We use the counts that the word “Rolex” appears in spam messages and messages that 
are not spam to find that p(Rolex) = 250/2000 = 0.125 and qg(Rolex) = 5/1000 = 0.005. 
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Because we are assuming that it is equally likely for an incoming message to be spam as it is 
not to be spam, we can estimate the probability that an incoming message containing the word 
“Rolex” is spam by 


p(Rolex) 0.125 _ 0.125 


r(Rolex) = = = 
p(Rolex) + q(Rolex) 0.125+0.005 0.130 


~ 0.962. 


Because 7 (Rolex) is greater than the threshold 0.9, we reject such messages as spam. < 


Detecting spam based on the presence of a single word can lead to excessive false positives 
and false negatives. Consequently, spam filters look at the presence of multiple words. For 
example, suppose that the message contains the words w; and w2. Let E; and E> denote the 
events that the message contains the words w, and w2, respectively. To make our computations 
simpler, we assume that E, and E?2 are independent events and that FE; | S and E> | S are 
independent events and that we have no prior knowledge regarding whether or not the message 
is spam. (The assumptions that F; and FE» are independent and that FE; | S and E> | S are 
independent may introduce some error into our computations; we assume that this error is small.) 
Using Bayes’ theorem and our assumptions, we can show (see Exercise 23) that p(S | £1 M E2), 
the probability that the message is spam given that it contains both w and wg, is 


p(E\ | S)p(E> | S) 
P(E, | S)p(E2 | S) + p(E1 | S)p(E2 |S) 


We estimate the probability p(S | Ey 9 E2) by 


p(S | Ei 0 Eo) = 


P(w1) p(w2) 
p(w1)p(w2) + (wig (wa) | 


r(wy,W2) = 


That is, r(w1, w2) estimates the probability that the message is spam, given that it contains the 
words w, and w2. When r(w1, w2) is greater than a preset threshold, such as 0.9, we determine 
that the message is likely spam. 


Suppose that we train a Bayesian spam filter on a set of 2000 spam messages and 1000 messages 
that are not spam. The word “stock” appears in 400 spam messages and 60 messages that are not 
spam, and the word “undervalued” appears in 200 spam messages and 25 messages that are not 
spam. Estimate the probability that an incoming message containing both the words “stock” and 
“undervalued” is spam, assuming that we have no prior knowledge about whether it is spam. 
Will we reject such messages as spam when we set the threshold at 0.9? 


Solution: Using the counts of each of these two words in messages known to be spam 
or known not to be spam, we obtain the following estimates: p(stock) = 400/2000 = 0.2, 
q(stock) = 60/1000 = 0.06, p(undervalued) = 200/2000 = 0.1, and gq(undervalued) = 
25/1000 = 0.025. Using these probabilities, we can estimate the probability that the message 
is spam by 


stock dervalued 
r(stock, undervalued) = dela 2 aaa alae 


p(stock) p(undervalued) + q(stock)q (undervalued) 


(0.2)(0.1) 


= @DO1) 4 0.06) (0.005) 73°: 


Because we have set the threshold for rejecting messages at 0.9, such messages will be rejected 
by the filter. < 


The more words we use to estimate the probability that an incoming mail message is spam, 
the better is our chance that we correctly determine whether it is spam. In general, if FE; is the 


Bayesian poisoning, the 
insertion of extra words to 
defeat spam filters, can 
use random or 
purposefully selected 
words. 


Exercises 


1. Suppose that E and F are events in a sample space and 
P(E) = 1/3, p(F) = 1/2, and p(E | F) = 2/5. Find 


p(F | E). 
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event that the message contains word w;, assuming that the number of incoming spam messages 
is approximately the same as the number of incoming messages that are not spam, and that the 
events E; | S are independent, then by Bayes’ theorem the probability that a message containing 
all the words wj, w2,..., we 1S spam is 


k k 
Hiei pC |S) 
PS (| Ei) = = 
= Fre 194 TT eT 


We can estimate this probability by 


i P(wi) 
13 p(wi) + eS qi) 


r(W1,W2,...,Wk) = 


For the most effective spam filter, we choose words for which the probability that each 
of these words appears in spam is either very high or very low. When we compute this value 
for a particular message, we reject the message as spam if r(w 1, w2,..., We) exceeds a preset 
threshold, such as 0.9. 

Another way to improve the performance of a Bayesian spam filter is to look at the prob- 
abilities that particular pairs of words appear in spam and in messages that are not spam. We 
then treat appearances of these pairs of words as appearance of a single block, rather than as 
the appearance of two separate words. For example, the pair of words “enhance performance” 
most likely indicates spam, while “operatic performance” indicates a message that is not spam. 
Similarly, we can assess the likelihood that a message is spam by examining the structure of a 
message to determine where words appear in it. Also, spam filters look at appearances of certain 
types of strings of characters rather than just words. For example, a message with the valid 
e-mail address of one of your friends is less likely to be spam (if not sent by a worm) than one 
containing an e-mail address that came from a country known to originate a lot of spam. There 
is an ongoing war between people who create spam and those trying to filter their messages out. 
This leads to the introduction of many new techniques to defeat spam filters, including inserting 
into spam messages long strings of words that appear in messages that are not spam, as well as 
including words inside pictures. The techniques we have discussed here are only the first steps 
in fighting this war on spam. 


six black balls. What is the probability that Ann picked 
a ball from the second box if she has selected an orange 
ball? 


. Suppose that F and F are events in a sample space and 
p(E) = 2/3, p(F) = 3/4, and p(F | EF) = 5/8. Find 
p(E | F). 


. Suppose that Frida selects a ball by first picking one of 
two boxes at random and then selecting a ball from this 
box at random. The first box contains two white balls and 
three blue balls, and the second box contains four white 
balls and one blue ball. What is the probability that Frida 
picked a ball from the first box if she has selected a blue 
ball? 


. Suppose that Ann selects a ball by first picking one of two 
boxes at random and then selecting a ball from this box. 
The first box contains three orange balls and four black 
balls, and the second box contains five orange balls and 


. Suppose that 8% of all bicycle racers use steroids, that 


a bicyclist who uses steroids tests positive for steroids 
96% of the time, and that a bicyclist who does not use 
steroids tests positive for steroids 9% of the time. What 
is the probability that a randomly selected bicyclist who 
tests positive for steroids actually uses steroids? 


6. When a test for steroids is given to soccer players, 98% 


of the players taking steroids test positive and 12% of the 
players not taking steroids test positive. Suppose that 5% 
of soccer players take steroids. What is the probability 
that a soccer player who tests positive takes steroids? 


. Suppose that a test for opium use has a 2% false positive 


rate and a 5% false negative rate. That is, 2% of peo- 
ple who do not use opium test positive for opium, and 
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10. 


11. 


*12, 


5% of opium users test negative for opium. Furthermore, 

suppose that 1% of people actually use opium. 

a) Find the probability that someone who tests negative 
for opium use does not use opium. 

b) Find the probability that someone who tests positive 
for opium use actually uses opium. 


. Suppose that one person in 10,000 people has a rare ge- 


netic disease. There is an excellent test for the disease; 
99.9% of people with the disease test positive and only 
0.02% who do not have the disease test positive. 


a) What is the probability that someone who tests posi- 
tive has the genetic disease? 

b) What is the probability that someone who tests nega- 
tive does not have the disease? 


. Suppose that 8% of the patients tested in a clinic are in- 


fected with HIV. Furthermore, suppose that when a blood 

test for HIV is given, 98% of the patients infected with 

HIV test positive and that 3% of the patients not infected 

with HIV test positive. What is the probability that 

a) a patient testing positive for HIV with this test is in- 
fected with it? 

b) a patient testing positive for HIV with this test is not 
infected with it? 

c) a patient testing negative for HIV with this test is in- 
fected with it? 

d) a patient testing negative for HIV with this test is not 
infected with it? 


Suppose that 4% of the patients tested in a clinic are in- 
fected with avian influenza. Furthermore, suppose that 
when a blood test for avian influenza is given, 97% of the 
patients infected with avian influenza test positive and 
that 2% of the patients not infected with avian influenza 
test positive. What is the probability that 


a) a patient testing positive for avian influenza with this 
test is infected with it? 

b) a patient testing positive for avian influenza with this 
test is not infected with it? 

c) apatient testing negative for avian influenza with this 
test is infected with it? 

d) a patient testing negative for avian influenza with this 
test is not infected with it? 


An electronics company is planning to introduce a new 
camera phone. The company commissions a marketing 
report for each new product that predicts either the success 
or the failure of the product. Of new products introduced 
by the company, 60% have been successes. Furthermore, 
70% of their successful products were predicted to be 
successes, while 40% of failed products were predicted 
to be successes. Find the probability that this new camera 
phone will be successful if its success has been predicted. 
A space probe near Neptune communicates with Earth us- 
ing bit strings. Suppose that in its transmissions it sends 
a | one-third of the time and a 0 two-thirds of the time. 
When a 0 is sent, the probability that it is received cor- 
rectly is 0.9, and the probability that it is received incor- 
rectly (as a 1) is 0.1. When a 1 is sent, the probability that 
it is received correctly is 0.8, and the probability that it is 
received incorrectly (as a 0) is 0.2. 


13. 


14. 


15. 


16. 


a) Find the probability that a 0 is received. 


b) Use Bayes’ theorem to find the probability that a 0 
was transmitted, given that a 0 was received. 


Suppose that E, Fi, Fo, and F3 are events from a 
sample space S$ and that Fi, Fo, and F3 are pair- 
wise disjoint and their union is S. Find p(F; | E) 
if p(E | Fi) = 1/8, p(E | F2)=1/4, p(E | F3) =1/6, 
D(F\) = 1/4, p(F2) = 1/4, and p(F3) = 1/2. 


Suppose that E, Fi, Fo, and F3 are events from a 
sample space S$ and that Fi, Fo, and F3 are pair- 
wise disjoint and their union is S. Find p(F2 | E) if 
P(E | Fi) = 2/7, p(B | Fo) = 3/8, p(E | F3) = 1/2, 
P(F\) = 1/6, p(F2) = 1/2, and p(F3) = 1/3. 


In this exercise we will use Bayes’ theorem to solve the 
Monty Hall puzzle (Example 10 in Section 7.1). Recall 
that in this puzzle you are asked to select one of three 
doors to open. There is a large prize behind one of the 
three doors and the other two doors are losers. After you 
select a door, Monty Hall opens one of the two doors you 
did not select that he knows is a losing door, selecting at 
random if both are losing doors. Monty asks you whether 
you would like to switch doors. Suppose that the three 
doors in the puzzle are labeled 1, 2, and 3. Let W be the 
random variable whose value is the number of the winning 
door; assume that p(W = k) = 1/3 for k = 1, 2, 3. Let 
M denote the random variable whose value is the number 
of the door that Monty opens. Suppose you choose door i. 


a) What is the probability that you will win the prize if 
the game ends without Monty asking you whether you 
want to change doors? 

b) Find p(M=j|W=k) for j=1,2,3 and k= 
1, 2, 3. 

c) Use Bayes’ theorem to find p(W = j | M =k) where 
i and j and k are distinct values. 


d) Explain why the answer to part (c) tells you whether 
you should change doors when Monty gives you the 
chance to do so. 


Ramesh can get to work in three different ways: by bicy- 
cle, by car, or by bus. Because of commuter traffic, there 
is a 50% chance that he will be late when he drives his 
car. When he takes the bus, which uses a special lane re- 
served for buses, there is a 20% chance that he will be 
late. The probability that he is late when he rides his bi- 
cycle is only 5%. Ramesh arrives late one day. His boss 
wants to estimate the probability that he drove his car to 
work that day. 


a) Suppose the boss assumes that there is a 1/3 chance 
that Ramesh takes each of the three ways he can get to 
work. What estimate for the probability that Ramesh 
drove his car does the boss obtain from Bayes’ theo- 
rem under this assumption? 


b) Suppose the boss knows that Ramesh drives 30% of 
the time, takes the bus only 10% of the time, and takes 
his bicycle 60% of the time. What estimate for the 
probability that Ramesh drove his car does the boss 
obtain from Bayes’ theorem using this information? 


*17, 


18. 


19. 


20. 


21. 


Expected Value and Variance 


Prove Theorem 2, the extended form of Bayes’ theo- 
rem. That is, suppose that E is an event from a sample 
space S and that F), Fo,..., F, are mutually exclusive 
events such that LJ!_, F; = S. Assume that p(E) 40 
and p(F;) 4 0 fori = 1,2,...,n. Show that 


D(E | Fj) p(Fj) 
YL P(E | Fi) pF) 


p(Fj | E) = 


[Hint: Use the fact that E = LJ!_,(E F;).] 

Suppose that a Bayesian spam filter is trained on a set of 
500 spam messages and 200 messages that are not spam. 
The word “exciting” appears in 40 spam messages and 
in 25 messages that are not spam. Would an incom- 
ing message be rejected as spam if it contains the word 
“exciting” and the threshold for rejecting spam is 0.9? 
Suppose that a Bayesian spam filter is trained on a set 
of 1000 spam messages and 400 messages that are not 
spam. The word “opportunity” appears in 175 spam mes- 
sages and 20 messages that are not spam. Would an in- 
coming message be rejected as spam if it contains the 
word “opportunity” and the threshold for rejecting a mes- 
sage is 0.9? 

Would we reject a message as spam in Example 4 


a) using just the fact that the word “undervalued” occurs 
in the message? 

b) using just the fact that the word “stock” occurs in the 
message? 

Suppose that a Bayesian spam filter is trained on a set 

of 10,000 spam messages and 5000 messages that are not 

spam. The word “enhancement” appears in 1500 spam 


22. 


23. 


7.4 Expected Value and Variance 477 


messages and 20 messages that are not spam, while the 
word “herbal” appears in 800 spam messages and 200 
messages that are not spam. Estimate the probability that 
a received message containing both the words “enhance- 
ment” and “herbal” is spam. Will the message be rejected 
as spam if the threshold for rejecting spam is 0.9? 


Suppose that we have prior information concerning 
whether a random incoming message is spam. In par- 
ticular, suppose that over a time period, we find that s 
spam messages arrive and fh messages arrive that are 
not spam. 

a) Use this information to estimate p(S), the probabil- 
ity that an incoming message is spam, and p(S), the 
probability an incoming message is not spam. 

b) Use Bayes’ theorem and part (a) to estimate the prob- 
ability that an incoming message containing the word 
w is spam, where p(w) is the probability that w occurs 
in a spam message and q(w) is the probability that w 
occurs in a message that is not spam. 

Suppose that E; and E> are the events that an incoming 

mail message contains the words w, and w3, respectively. 

Assuming that E; and E> are independent events and that 

E, | S and E> | S are independent events, where S is the 

event that an incoming message is spam, and that we have 

no prior knowledge regarding whether or not the message 
is spam, show that 


D(S | E, 0 Ep) 


_ p(E1 | $)p(E2 | 8) 
p(E\ | 8)p(E2 | S) + p(Er | S)p(E2 | 3)" 


Introduction 


Links 


The expected value of a random variable is the sum over all elements in a sample space of the 
product of the probability of the element and the value of the random variable at this element. 
Consequently, the expected value is a weighted average of the values of a random variable. The 
expected value of a random variable provides a central point for the distribution of values of 
this random variable. We can solve many problems using the notion of the expected value of a 
random variable, such as determining who has an advantage in gambling games and computing 
the average-case complexity of algorithms. Another useful measure of a random variable is its 
variance, which tells us how spread out the values of this random variable are. We can use the 
variance of a random variable to help us estimate the probability that a random variable takes 
values far removed from its expected value. 


Expected Values 


Many questions can be formulated in terms of the value we expect a random variable to take, 
or more precisely, the average value of a random variable when an experiment is performed a 
large number of times. Questions of this kind include: How many heads are expected to appear 
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when a coin is flipped 100 times? What is the expected number of comparisons used to find an 
element in a list using a linear search? To study such questions we introduce the concept of the 
expected value of a random variable. 


The expected value, also called the expectation or mean, of the random variable X on the 
sample space S is equal to 


E(X) = ) p(s) X(s). 


ses 


The deviation of X ats € Sis X(s) — E(X), the difference between the value of X and the 
mean of X. 


Note that when the sample space S$ has n elements S$ = {x1,x2,...,%n}, E(X) = 


yy pax 


Remark: When there are infinitely many elements of the sample space, the expectation is de- 
fined only when the infinite series in the definition is absolutely convergent. In particular, the 
expectation of a random variable on an infinite sample space is finite if it exists. 


Expected Value of a Die Let X be the number that comes up when a fair die is rolled. What 
is the expected value of X? 


Solution: The random variable X takes the values 1, 2, 3, 4, 5, or 6, each with probability 1/6. 
It follows that 


1 1 1 1 1 1 21 7 < 


6.6 eee ie eae ee ied oh 6 a ee 
a ae te OS = 


A fair coin is flipped three times. Let S be the sample space of the eight possible outcomes, and 
let X be the random variable that assigns to an outcome the number of heads in this outcome. 
What is the expected value of X? 


Solution: In Example 10 of Section 7.2 we listed the values of X for the eight possible outcomes 


when a coin is flipped three times. Because the coin is fair and the flips are independent, the 
probability of each outcome is 1/8. Consequently, 


1 
E(X) = re) + X (HHT) + X (HTH) + X (THH) + X (ITH) 
+ X (THT) + X (HTT) + X(TTT)] 


12 
= gG+24+242+14+14+1+0) == 


WN] WwW Cole 


Consequently, the expected number of heads that come up when a fair coin is flipped three times 


is 3/2. 4 


When an experiment has relatively few outcomes, we can compute the expected value of 
a random variable directly from its definition, as was done in Example 2. However, when an 
experiment has a large number of outcomes, it may be inconvenient to compute the expected 
value of a random variable directly from its definition. Instead, we can find the expected value 
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of a random variable by grouping together all outcomes assigned the same value by the random 
variable, as Theorem | shows. 


THEOREM 1 If X is a random variable and p(X = r) is the probability that X = r, so that p(X =r) = 
Dses,xX(s)=r P(S), then 


EO) — ep — 1) 


rex(S) 


Proof: Suppose that X is a random variable with range X(S), and let p(X =r) be the proba- 
bility that the random variable X takes the value 7. Consequently, p(X = r) is the sum of the 
probabilities of the outcomes s such that X(s) = r. It follows that 


E(X)= > p(X =ryr. q 


rex(S) 


Example 3 and the proof of Theorem 2 will illustrate the use of this formula. In Example 
3 we will find the expected value of the sum of the numbers that appear on two fair dice when 
they are rolled. In Theorem 2 we will find the expected value of the number of successes when 
n Bernoulli trials are performed. 


EXAMPLE 3 What is the expected value of the sum of the numbers that appear when a pair of fair dice is 
rolled? 


Solution: Let X be the random variable equal to the sum of the numbers that appear when a 
pair of dice is rolled. In Example 12 of Section 7.2 we listed the value of X for the 36 out- 
comes of this experiment. The range of X is {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}. By Example 12 of 
Section 7.2 we see that 


P(X = 2) = p(X = 12) = 1/36, 

p(X = 3) = p(X = 11) = 2/36 = 1/18, 
p(X =4) = p(X = 10) = 3/36 = 1/12, 
p(X =5) = p(X =9) = 4/36 = 1/9, 
p(X = 6) = p(X = 8) = 5/36, 

p(X = 7) = 6/36 = 1/6. 


Substituting these values in the formula, we have 


1 1 1 1 D 1 

Bix) =2: As ’ 7. 
(Xx) a ig? mt? gre 36° 6 
5 1 1 1 

10- 11-—412-— 

+8 ae Po + 10 pt ig * 36 


THEOREM 2 — The expected number of successes when n mutually independent Bernoulli trials are per- 
formed, where p is the probability of success on each trial, is np. 
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Proof: Let X be the random variable equal to the number of successes in n trials. By 
Theorem 2 of Section 7.2 we see that p(X = k) = C(n, k)pkq"-*. Hence, we have 


n 
E(X)= » kp(X =k) by Theorem 1 
k=1 
n 
a = kC(n, Kip ga’ by Theorem 2 in Section 7.2 
k=1 
n 
= >. nC(n—1,k — jpg? * by Exercise 21 in Section 6.4 
k=1 


n 


=np ye Can-1,k- ie tg factoring np from each term 


k=1 
n—-1 

= np > Cia — 1, Dpigrty shifting index of summation with j = k — 1 
j=0 

=np(pt+ ay by the binomial theorem 

=np. because p+q =1 


This completes the proof because it shows that the expected number of successes in n mutually 
independent Bernoulli trials is np. d 


We will also show that the hypothesis that the Bernoulli trials are mutually independent in 
Theorem 2 is not necessary. 
Linearity of Expectations 
Theorem 3 tells us that expected values are linear. For example, the expected value of the sum 


of random variables is the sum of their expected values. We will find this property exceedingly 
useful. 


THEOREM 3 If X;,i = 1,2,...,n withn a positive integer, are random variables on S, and if a and b are 
real numbers, then 


(i) E(X1 + Xo +--+ + Xn) = E(X1) + E(X2) +--+ + E(Xn) 
(ii) E(aX +b) =aE(X) +b. 


Proof: Part (i) follows for n = 2 directly from the definition of expected value, because 


E(X1 + Xo) = )> p(s)(Xi(s) + X2(s)) 


ses 
= Y- p(s) X1(s) + )> p(s) Xa(s) 
seS seS 


= E(X1) + E(X2). 


The case for n random variables follows easily by mathematical induction using the case of two 
random variables. (We leave it to the reader to complete the proof.) 


EXAMPLE 4 


EXAMPLE 5 


EXAMPLE 6 
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To prove part (ii), note that 


E(aX + b) = Vises P(S)(aX (s) + b) 


=a ices P(S)X (8) +b Di ccs p(s) 
= aE(X) +b because > ,.5 p(s) = 1. 


Examples 4 and 5 illustrate how to use Theorem 3. 


Use Theorem 3 to find the expected value of the sum of the numbers that appear when a pair of 
fair dice is rolled. (This was done in Example 3 without the benefit of this theorem.) 


Solution: Let X; and X be the random variables with X,((i, j/)) =i and X2((i, j)) = j, so 
that X, is the number appearing on the first die and X2 is the number appearing on the second die. 
It is easy to see that E(X,) = E(X2) = 7/2 because both equal (1 +2+3+4+4+5+46)/6= 
21/6 = 7/2. The sum of the two numbers that appear when the two dice are rolled is the sum 
X, + X2. By Theorem 3, the expected value of the sum is E(X, + X2) = E(X1) + E(X2) = 
7/2+7/2 =7. < 


In the proof of Theorem 2 we found the expected value of the number of successes when n 
independent Bernoulli trials are performed, where p is the probability of success on each trial 
by direct computation. Show how Theorem 3 can be used to derive this result where the Bernoulli 
trials are not necessarily independent. 


Solution: Let X; be the random variable with X;((t},f,...,f))=1 if t; 1s a suc- 
cess and Xj((t, f2,...,%)) =0 if t; is a failure. The expected value of X; is E(Xj) = 
1-p+0-(—p)=p fori =1,2,...,n. Let X¥ = X;+ X2+---+ Xy, so that X counts 
the number of successes when these n Bernoulli trials are performed. Theorem 3, applied to the 
sum of n random variables, shows that F(X) = E(X,) + E(X2)+---+ E(X;) = np. < 


We can take advantage of the linearity of expectations to find the solutions of many seemingly 
difficult problems. The key step is to express a random variable whose expectation we wish to 
find as the sum of random variables whose expectations are easy to find. Examples 6 and 7 
illustrate this technique. 


Expected Value in the Hatcheck Problem A new employee checks the hats of n people at a 
restaurant, forgetting to put claim check numbers on the hats. When customers return for their 
hats, the checker gives them back hats chosen at random from the remaining hats. What is the 
expected number of hats that are returned correctly? 


Solution: Let X be the random variable that equals the number of people who receive the correct 


hat from the checker. Let X; be the random variable with X; = 1 if the ith person receives the 
correct hat and X; = 0 otherwise. It follows that 


X =X, + X24+---+ Xp. 
Because it is equally likely that the checker returns any of the hats to this person, it follows that 
the probability that the ith person receives the correct hat is 1/n. Consequently, by Theorem 1, 


for all i we have 


E(X;) = 1- p(X = 1) + 0- p(X = 0) =1-1/n+0=1/n. 
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By the linearity of expectations (Theorem 3), it follows that 
E(X) = E(X1) + E(X2) +--+ + E(X,) = n- 1/n=1. 


Consequently, the average number of people who receive the correct hat is exactly 1. Note 
that this answer is independent of the number of people who have checked their hats! 
(We will find an explicit formula for the probability that no one receives the correct hat in 
Example 4 of Section 8.6.) 


Expected Number of Inversions in a Permutation The ordered pair (i, j) is called an 
inversion in a permutation of the first n positive integers if i < j but j precedes i in the 
permutation. For instance, there are six inversions in the permutation 3, 5, 1, 4, 2; these 
inversions are 


(1, 3), (1, 5), (2, 3), (2, 4), (2, 5), (4, 5). 


Let J;, ; be the random variable on the set of all permutations of the first n positive integers with 
J;,; = 1 if (, j) is an inversion of the permutation and J;,; = 0 otherwise. It follows that if X 
is the random variable equal to the number of inversions in the permutation, then 


l<i<j<n 


Note that it is equally likely for i to precede j in a randomly chosen permutation as it is for j to 
precede 7. (To see this, note that there are an equal number of permutations with each of these 
properties.) Consequently, for all pairs i and j we have 


E(ij) =1- pi =1) +0- pj = 0) =1-1/24+0=1/2. 


Because there are (5) pairs i and j with 1 <i < j <n and by the linearity of expectations 


(Theorem 3), we have 


EX)= So Ej) = (5) ; = a 


l<i<j<n 


It follows that there are an average of n(n — 1)/4 inversions in a permutation of the first n 
positive integers. 


Average-Case Computational Complexity 


Computing the average-case computational complexity of an algorithm can be interpreted as 
computing the expected value of a random variable. Let the sample space of an experiment be 
the set of possible inputs aj, j = 1,2,...,n, and let X be the random variable that assigns 
to a; the number of operations used by the algorithm when given a; as input. Based on our 
knowledge of the input, we assign a probability p(aj;) to each possible input value a;. Then, 
the average-case complexity of the algorithm is 


E(X) = > p(aj)X (aj). 


j=l 


This is the expected value of X. 


EXAMPLE 8 


EXAMPLE 9 
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Finding the average-case computational complexity of an algorithm is usually much more 
difficult than finding its worst-case computational complexity, and often involves the use of 
sophisticated methods. However, there are some algorithms for which the analysis required 
to find the average-case computational complexity is not difficult. For instance, in Example 8 
we will illustrate how to find the average-case computational complexity of the linear search 
algorithm under different assumptions concerning the probability that the element for which we 
search is an element of the list. 


Average-Case Complexity of the Linear Search Algorithm We are given a real number x 
and a list of n distinct real numbers. The linear search algorithm, described in Section 3.1, 
locates x by successively comparing it to each element in the list, terminating when x is located 
or when all the elements have been examined and it has been determined that x is not in the 
list. What is the average-case computational complexity of the linear search algorithm if the 
probability that x is in the list is p and it is equally likely that x is any of the n elements in the 
list? (There are n + | possible types of input: one type for each of the n numbers in the list and 
a last type for numbers not in the list, which we treat as a single input.) 


Solution: In Example 4 of Section 3.3 we showed that 2i + 1 comparisons are used if x equals 
the ith element of the list and, in Example 2 of Section 3.3, we showed that 2n + 2 comparisons 
are used if x is not in the list. The probability that x equals a;, the ith element in the list, is 
p/n, and the probability that x is not in the list is g = 1 — p. It follows that the average-case 
computational complexity of the linear search algorithm is 


E= 


3 2) 2n+ 1 
== P 4 FOP 4 ont 2g 


+4. 
n 
=2G45+---+Qn+1)+Qn+2q 


es Fun $1)? = 1 nog 


= p(n+2)+ n+ 2)q. 


(The third equality follows from Example 2 of Section 5.1.) For instance, when x is guaranteed 
to be in the list, we have p = | (so the probability that x = a; is 1/n for each 7) and gq = 0. 
Then E =n + 2, as we showed in Example 4 in Section 3.3. 

When p, the probability that x is in the list, is 1/2, it follows that g = 1 — p = 1/2, 
so E = (n+2)/2+n+1= 3n+4)/2. Similarly, if the probability that x is in the list 
is 3/4, we have p = 3/4 and gq = 1/4,so E = 3(n+ 2)/44+ (n+ 1)/2 = (Sn + 8)/4. 

Finally, when x is guaranteed not to be in the list, we have p = 0 and qg = 1. It follows that 
E = 2n + 2, which is not surprising because we have to search the entire list. 


Example 9 illustrates how the linearity of expectations can help us find the average-case 
complexity of a sorting algorithm, the insertion sort. 


Average-Case Complexity of the Insertion Sort What is the average number of comparisons 
used by the insertion sort to sort n distinct elements? 


Solution: We first suppose that X is the random variable equal to the number of comparisons used 
by the insertion sort (described in Section 3.1) to sort a list aj, a2, ..., dy of n distinct elements. 
Then E(X) is the average number of comparisons used. (Recall that at step i fori = 2,...,n, 
the insertion sort inserts the ith element in the original list into the correct position in the sorted 
list of the first i — 1 elements of the original list.) 
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We let X; be the random variable equal to the number of comparisons used to insert a; into 
the proper position after the first i — 1 elements a), a2, ..., aj—; have been sorted. Because 


X=X.+Xate+ Xn, 
we can use the linearity of expectations to conclude that 
E(X) = E(X2. + X3+-++++ Xn) = E(X2) + E(X3) +--+ + E(Xn). 


To find E(X;) for i = 2,3,...,n, let pj(k) denote the probability that the largest of the 
first 7 elements in the list occurs at the kth position, that is, that max(a, a2,...,aj;) = ak, 
where 1 < k < j. Because the elements of the list are randomly distributed, it is equally likely 
for the largest element among the first j elements to occur at any position. Consequently, 
pj(k) = 1/j. If X;(k) equals the number of comparisons used by the insertion sort if a; is 
inserted into the kth position in the list once a), a2,...,a;—-, have been sorted, it follows 
that X;(k) = k. Because it is possible that a; is inserted in any of the first i positions, we 
find that 


i i 
1 1 1 witty) @+1 
E(Xi) = ) 0 pik) Xik) = Dok =>: k= 5 = 
k=1 k=1 k=1 


It follows that 


n n p41 yt 

BOs) EC) = 5" 5 =5 i 
i=2 i=2 j=3 
lan+1)(n+2 1 n2+3n—4 
2 2 2 4 


To obtain the third of these equalities we shifted the index of summation, setting j = i+ 1. 
To obtain the fourth equality, we used the formula )°;"_, k = m(m + 1)/2 (from Table 2 in 
Section 2.4) with m =n-+ 1, subtracting off the missing terms with 7 = 1 and j = 2. We 
conclude that the average number of comparisons used by the insertion sort to sort n elements 
equals (n? + 3n — 4)/4, which is @(n”). < 


The Geometric Distribution 


We now turn our attention to a random variable with infinitely many possible outcomes. 


Suppose that the probability that a coin comes up tails is p. This coin is flipped repeatedly until 
it comes up tails. What is the expected number of flips until this coin comes up tails? 


Solution: We first note that the sample space consists of all sequences that begin with any 
number of heads, denoted by H, followed by a tail, denoted by T. Therefore, the sam- 
ple space is the set {J HT, HHT, HHHT, HHHHT, ...}. Note that this is an infinite sample 
space. We can determine the probability of an element of the sample space by noting that 
the coin flips are independent and that the probability of a head is 1 — p. Therefore, p(T) = p, 
p(HT) = (1 — p)p, p(HHT)= (1 — p)* p, and in general the probability that the coin is flipped 
n times before a tail comes up, that is, thatn — 1 heads come up followed by atail, is (1 — p)"~! p. 
(Exercise 14 asks for a verification that the sum of the probabilities of the points in the sample 
space is 1.) 


DEFINITION 2 


THEOREM 4 
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Now let X be the random variable equal to the number of flips in an element in the 
sample space. That is, X(T) = 1, X(HT)= 2, X(HHT)= 3, and so on. Note that p(X = j) = 
(1 — p)/~! p. The expected number of flips until the coin comes up tails equals E(X). 

Using Theorem 1, we find that 


i 
P 


ems 


EX) = hj PX == Id py 'p =p Id — py =p = 


j=l j=l j=l 


[The third equality in this chain follows from Table 2 in Section 2.4, which tells us 
that pa jd — p)i-! =1/ —(1— p))* = 1/p?.] It follows that the expected num- 
ber of times the coin is flipped until tails comes up is 1/p. Note that when the 
coin is fair we have p= 1/2, so the expected number of flips until it comes up tails 
is 1/(1/2) = 2. 


The random variable X that equals the number of flips expected before a coin comes up 


tails is an example of a random variable with a geometric distribution. 


A random variable X has a geometric distribution with parameter p if p(X =k) = 
= Diep fork = 1,2,3,..., where p is areal number with 0 < p < 1. 


Geometric distributions arise in many applications because they are used to study the time 
required before a particular event happens, such as the time required before we find an object 
with a certain property, the number of attempts before an experiment succeeds, the number of 
times a product can be used before it fails, and so on. 

When we computed the expected value of the number of flips required before a coin comes 
up tails, we proved Theorem 4. 


If the random variable X has the geometric distribution with parameter p, then E(X) = 1/p. 


Independent Random Variables 
We have already discussed independent events. We will now define what it means for two random 
variables to be independent. 
The random variables X and Y on a sample space S are independent if 
p(X =r, and ¥ = 19) = p(X =r) - p(Y = 7), 
or in words, if the probability that X = r; and Y = r2 equals the product of the probabilities 
that X = r; and Y = ry, for all real numbers r; and rp. 
Are the random variables X; and X2 from Example 4 independent? 


Solution: Let S = {1,2,3,4,5, 6}, and let i € S and j € S. Because there are 36 possible 
outcomes when the pair of dice is rolled and each is equally likely, we have 


p(X, =i and X2 = j) = 1/36. 
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Furthermore, p(X; =i) = 1/6 and p(X2 = j) = 1/6, because the probability that i appears 
on the first die and the probability that 7 appears on the second die are both 1/6. It follows that 


: ‘ 1 ; : 
p(X, =iand X2=j)= 36 and p(X; =i)p(X2= j)= 


so X; and X2 are independent. < 


EXAMPLE 12 — Show that the random variables X; and X = X; + X2, where X, and X2 are as defined in 
Example 4, are not independent. 


Solution: Note that p(X; = | and X = 12) = 0, because X; = 1 means the number appear- 
ing on the first die is 1, which implies that the sum of the numbers appearing on the two 
dice cannot equal 12. On the other hand, p(X; = 1) = 1/6 and p(X = 12) = 1/36. Hence 
p(X, = land X = 12) ¥ p(X; = 1)- p(X = 12). This counterexample shows that X; and X 
are not independent. < 


The expected value of the product of two independent random variables is the product of 
their expected values, as Theorem 5 shows. 


THEOREM 5 _ IfX and Y are independent random variables ona sample space S, then E(XY) = E(X)E(Y). 


Proof: To prove this formula, we use the key observation that the event XY = r is the disjoint 
union of the events X = rj and Y =r over all r; € X(S) and r2 € Y(S) with r = rjro. We 


have 
E(XY)= > r- p(XY =r) by Theorem 1 
rexY(S) 
= a riro: p(X =r, andY = 19) expressing XY =r as a disjoint union 
ry} EX(S),r2EY(S) 
= 2 > rir: p(X =r, and Y =r) using a double sum to order the terms 


ryEX (S) 2€Y(S) 


= 2 > rir2: p(X =r): pY =1r2) by the independence of X and Y 
rjEX (S) 2€Y(S) 


= > (ri -p(X =r): > 2°: pVY= r2)) by factoring out ry - p(X = r}) 


rpeX(S) r2€Y(S) 
= » ry: p(X =r1)- EY) by the definition of E(Y) 
ryeX(S) 
= E(Y)( > ry: p(X =r)) by factoring out E(Y) 
ryeX(S) 
= E(Y)E(X) by the definition of F(X) 


We complete the proof by noting that E(Y)E(X) = E(X)E(Y), which is a consequence of the 
commutative law for multiplication. | 


Note that when X and Y are random variables that are not independent, we cannot conclude 
that E(XY) = E(X)E(Y), as Example 13 shows. 


EXAMPLE 13 


Links 


DEFINITION 4 


THEOREM 6 
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Let X and Y be random variables that count the number of heads and the number of tails when 
a coin is flipped twice. Because p(X = 2) = 1/4, p(X = 1) = 1/2, and p(X = 0) = 1/4, by 
Theorem | we have 

1 
2. 


A similar computation shows that E(Y) = 1. We note that X Y = 0 when either two heads and 
no tails or two tails and no heads come up and that X Y = | when one head and one tail come 
up. Hence, 


1 1 
EQ) =2)— 41+ 240. = 1, 
(XxX) ras + q 


1 
E(XY)=1-~+0--= 
(XY) ee 


Nile 


It follows that 
E(XY) 4 E(X)E(Y). 


This does not contradict Theorem 5 because X and Y are not independent, as the reader should 
verify (see Exercise 16). 


Variance 


The expected value of a random variable tells us its average value, but nothing about how 
widely its values are distributed. For example, if X and Y are the random variables on the set 
S = {1,2,3,4,5, 6}, with X¥(s) = 0 for alls € Sand Y(s) = —1ifs € {1, 2,3} and Y(s) = 1 
if s € {4,5, 6}, then the expected values of X and Y are both zero. However, the random 
variable X never varies from 0, while the random variable Y always differs from 0 by 1. The 
variance of a random variable helps us characterize how widely a random variable is distributed. 
In particular, it provides a measure of how widely X is distributed about its expected value. 


Let X be a random variable on a sample space S. The variance of X, denoted by V(X), is 


V(X) = )0(X(s) — E(X))’ Gs). 


ses 


That is, V(X) is the weighted average of the square of the deviation of X. The standard 
deviation of X, denoted o (X), is defined to be / V(X). 


Theorem 6 provides a useful simple expression for the variance of a random variable. 


If X is arandom variable on a sample space S, then V(X) = E(X?) _ (BOCE. 


Proof: Note that 


V(X) = SOX) — E(X*))" p(s) 


ses 
= D5 X(s)" pls) — 2E(X) D> X (8) ps) + EX)? YS p(s) 
ses ses ses 


= E(X”) —2E(X)E(X) + E(X)? 
= E(X”) — E(X). 


We have used the fact that }° 5 p(s) = 1 in the next-to-last step. | 
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We can use Theorems 3 and 6 to derive an alternative formula for V (X) that provides some 
insight into the meaning of the variance of a random variable. 


COROLLARY 1 __ If X isarandom variable on a sample space S and E(X) = j, then V(X) = E((X — 1)”). 


vis the Greek letter mu. Proof: If X is a random variable with E(X) = j, then 
E((X — w)*) = E(X* — 2uX +p’) expanding (X — 1)? 
= E(X*) — E(2uX) + E(1”) _ by part (i) of Theorem 3 


= E(X?) — 2WE(X) + E(u”) by part (ii) of Theorem 3, noting that yz is a constant 


= E(X?) — 2WE(X) + ie as E(.2) = x2, because jx” is a constant 
= E(X?) = dye + pe because E(X) = 
= E(X’) = Ve simplifying 
= V(X) by Theorem 6 and noting that E(X) = yp. 
This completes the proof. | 


Corollary | tells us that the variance of a random variable X is the expected value of the 
square of the difference between X and its own expected value. This is commonly expressed 
as saying that the variance of X is the mean of the square of its deviation. We also say that the 
standard deviation of X is the square root of the mean of the square of its deviation (often read 
as the “root mean square” of the deviation). 

We now compute the variance of some random variables. 


EXAMPLE14 What is the variance of the random variable X with X(t) = 1 if a Bernoulli trial is a success 
and X(t) = O if it is a failure, where p is the probability of success and q is the probability of 
failure? 


Extra ~% Solution: Because X takes only the values 0 and 1, it follows that X 2(t) = X(t). Hence, 
Examples 


V(X) = E(X*) — E(X)’ = p— p? = p( — p) = pa. 4 


EXAMPLE 15 Variance of the Value of a Die What is the variance of the random variable X, where X is 
the number that comes up when a fair die is rolled? 


Solution: We have V(X) = E(X”) — E(X)*. By Example 1 we know that E(X) = 7/2. To find 


EC?) note that X2 takes the values i2, i=1,2,...,6, each with probability 1/6. It follows 
that 


1 91 
E(X?) = ZU +P + +4 45° +6) =i 


EXAMPLE 16 
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Links 
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We conclude that 


2 
vin = 2 - (3) _ 3 < 


What is the variance of the random variable X ((i, j)) = 2i, where i is the number appearing 
on the first die and 7 is the number appearing on the second die, when two fair dice are rolled? 


Solution: We will use Theorem 6 to find the variance of X. To do so, we need to find the 
expected values of X and X*. Note that because p(X = k) is 1/6 for k = 2, 4, 6, 8, 10, 12 and 
is 0 otherwise, 


E(X) = (2+44+6+8+4 10+ 12)/6=7, 
and 
E(X’) = (27 +47 + 6 4+ 87 + 10° + 127) /6 = 182/3. 
It follows from Theorem 6 that 
V(X) = E(X’) — E(X)* = 182/3 — 49 = 35/3. 4 


Another useful property is that the variance of the sum of two or more independent random 
variables is the sum of their variances. The formula that expresses this property is known as 
Bienaymé’s formula, after Irenée-Jules Bienaymé, the French mathematician who discovered it 
in 1853. Bienaymé’s formula is useful for computing the variance of the result of n independent 
Bernoulli trials, for instance. 


BIENAYME’S FORMULA If X and Y are two independent random variables on 
a sample space S, then V(X + Y) = V(X)+ V(Y). Furthermore, if X;,i = 1,2,...,n, 
with n a positive integer, are pairwise independent random variables on S, then 
Via ae Xo es A) VO) eV Xoo VX): 


IRENEE-JULES BIENAYME (1796-1878) Bienaymé, born in Paris, moved with his family to Bruges in 
1803 when his father became a government administrator. Bienaymé attended the Lycée impérial in Bruges, 
and when his family returned to Paris in 1811, the Lycée Louis-le-Grand. As a teenager, he helped defend Paris 
during the 1814 Napoleonic Wars; in 1815, he became a student at the Ecole Polytechnique. In 1816 he joined 
the Ministry of Finances to help support his family. In 1819, he left the civil service, taking a job lecturing 
mathematics at the Académie militaire de Saint-Cyr. Unhappy with conditions there, he soon returned to the 
Ministry of Finances. He attained the position of inspector general, remaining until forced to retire in 1848 
for political reasons. He was able to return as inspector general in 1850, but he retired a second time in 1852. 
In 1851 he briefly was professor at the Sorbonne and also served as an expert statistician for Napoleon III. 


Bienaymé was one of the founders of the Société Mathématique de France, and in 1875 was its president. 

Bienaymé was noted for his ingenuity, but his papers frustrated readers by omitting important proofs. He published sparsely, 
often in obscure journals. However, he made important contributions to probability and statistics, and to their applications to the 
social sciences and to finance. Among his important contributions are the Bienaymé-Chebyshev inequality, which provides a simple 
proof of the law of large numbers, a generalization of Laplace’s least square method, and Bienaymé’s formula for the variance of a 
sum of random variables. He studied the extinction of aristocratic families, declining despite general population growth. Bienaymé 
was a skilled linguist; he translated the works of Chebyshev, a close friend, from Russian to French. It has been suggested that his 
relative obscurity results from his modesty, his lack of interest in asserting the priority of his discoveries, and the fact that his work 
was often ahead of its time. He and his brother married two sisters who were daughters of a family friend. Bienaymé and his wife 
had two sons and three daughters. 
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EXAMPLE 17 


EXAMPLE 18 


Proof: From Theorem 6, we have 
V(X +Y) = E(X+Y)) - E(X+¥)’. 
It follows that 
V(X + Y) = E(X* +2XY + Y’) — (E(X) + E(¥))* 
= PO) 4 DENY) + BO) S EY = TERY) = 20. 
Because X and Y are independent, by Theorem 5 we have E(XY) = E(X)E(Y). It follows that 


V(X + Y) = (E(X?) — E(X)*) + (E(Y”) — E(Y)?) 
= V(X)+ V(Y). 


We leave the proof of the case for n pairwise independent random variables to the reader 
(Exercise 34). Such a proof can be constructed by generalizing the proof we have given for the 
case for two random variables. Note that it is not possible to use mathematical induction in a 
straightforward way to prove the general case (see Exercise 33). 4 


Find the variance and standard deviation of the random variable X whose value when two fair 
dice are rolled is X ((i, j)) = i+ j, where is the number appearing on the first die and j is the 
number appearing on the second die. 


Solution: Let X, and X2 be the random variables defined by X,((i, 7)) =i and 
X2((i, j)) = J for a roll of the dice. Then X = X; + X2, and X; and X> are independent, as 
Example 11 showed. From Theorem 7 it follows that V(X) = V(X) + V(X2). A sim- 
ple computation as in Example 16, together with Exercise 29 in the Supplementary 
Exercises, tells us that V(X,) = V(X2) = 35/12. Hence, V(X) = 35/12 + 35/12 = 35/6 
and 0 (X) = /35/6. < 


We will now find the variance of the random variable that counts the number of successes 
when n independent Bernoulli trials are carried out. 


What is the variance of the number of successes when n independent Bernoulli trials are per- 
formed, where, on each trial, p is the probability of success and g is the probability of failure? 


Solution: Let X; be the random variable with X;((t, fo,...,t))) = 1 if trial f; is a 
success and X;((t), fo,...,t,)) =O if trial 4; is a failure. Let X¥ = X; + X2+---+ Xp. 
Then X counts the number of successes in the n trials. From Theorem 7 it follows that V(X) = 
V(X) + V(X2) +--- + V(X,,). Using Example 14 we have V (X;) = pq fori = 1,2,...,n. 
It follows that V(X) = npq. < 


seq PAFNUTY LVOVICH CHEBYSHEV (1821-1894) Chebyshev was born into the gentry in Okatovo, Russia. 


| His father was a retired army officer who had fought against Napoleon. In 1832 the family, with its nine children, 
| moved to Moscow, where Pafnuty completed his high school education at home. He entered the Department of 


Physics and Mathematics at Moscow University. As a student, he developed a new method for approximating 
the roots of equations. He graduated from Moscow University in 1841 with a degree in mathematics, and he 
continued his studies, passing his master’s exam in 1843 and completing his master’s thesis in 1846. 
Chebyshev was appointed in 1847 to a position as an assistant at the University of St. Petersburg. He wrote 
and defended a thesis in 1847. He became a professor at St. Petersburg in 1860, a position he held until 1882. 
His book on the theory of congruences written in 1849 was influential in the development of number theory. His 


work on the distribution of prime numbers was seminal. He proved Bertrand’s conjecture that for every integer n > 3, there is a 
prime between n and 2n — 2. Chebyshev helped develop ideas that were later used to prove the prime number theorem. Chebyshev’s 
work on the approximation of functions using polynomials is used extensively when computers are used to find values of functions. 
Chebyshev was also interested in mechanics. He studied the conversion of rotary motion into rectilinear motion by mechanical 
coupling. The Chebyshev parallel motion is three linked bars approximating rectilinear motion. 
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Chebyshev’s Inequality 
How likely is it that a random variable takes a value far from its expected value? Theorem 8, 
called Chebyshev’s inequality, helps answer this question by providing an upper bound on the 


probability that the value of a random variable differs from the expected value of the random 
variable by more than a specified amount. 


CHEBYSHEV’S INEQUALITY Let X be a random variable on a sample space S with 
probability function p. If r is a positive real number, then 


p(|\X(s) — E(X)| =r) < V(X)/r?. 


Proof: Let A be the event 
A={s €S||X(s) — E(X)| =r}. 
What we want to prove is that p(A) < V(X)/r?. Note that 


V(X) = \(X(s) — E(X) p(s) 


ses 
= D\(X(8) — EXO) p(s) + 90 (X(s) — E(X))? PG). 
seA séA 


The second sum in this expression is nonnegative, because each of its summands is nonnegative. 
Also, because for each element s in A, (X (s) — E(X))* > r?, the first sum in this expression is at 
least )ooc4 r? p(s). Hence, V(X) > as r? p(s) =r? p(A). It follows that V(X)/r? > p(A), 
so p(A) < V(X)/r?, completing the proof. | 


Deviation from the Mean when Counting Tails Suppose that X is the random variable 
that counts the number of tails when a fair coin is tossed n times. Note that X is the number 
of successes when n independent Bernoulli trials, each with probability of success 1/2, are 
performed. It follows that E(X) = n/2 (by Theorem 2) and V(X) = n/4 (by Example 18). 
Applying Chebyshev’s inequality with r = ./n shows that 


p(\X(s) —n/2| = Vn) < (n/4)/G/n)? = 1/4. 


Consequently, the probability is no more than 1/4 that the number of tails that come up when a 
fair coin is tossed n times deviates from the mean by more than ,/n. 


Chebyshev’s inequality, although applicable to any random variable, often fails to provide 
a practical estimate for the probability that the value of a random variable exceeds its mean by 
a large amount. This is illustrated by Example 20. 


Let X be the random variable whose value is the number appearing when a fair die is rolled. 
We have E(X) = 7/2 (see Example 1) and V(X) = 35/12 (see Example 15). Because the 
only possible values of X are 1, 2, 3, 4,5, and 6, X cannot take a value more than 5/2 from its 
mean, E(X) = 7/2. Hence, p(|X — 7/2| > r) = 0 if r > 5/2. By Chebyshev’s inequality we 
know that p(|X — 7/2| > r) < (35/12)/r?. 

For example, when r = 3, Chebyshev’s inequality tells us that p(|X — 7/2| > 3) < 
(35/12)/9 = 35/108 © 0.324, which is a poor estimate, because p(|X — 7/2| >3)=0. <4 
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Exercises 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


. What is the expected number of heads that come up when 


a fair coin is flipped five times? 


. What is the expected number of heads that come up when 


a fair coin is flipped 10 times? 


. What is the expected number of times a 6 appears when 


a fair die is rolled 10 times? 


. A coin is biased so that the probability a head comes up 


when it is flipped is 0.6. What is the expected number of 
heads that come up when it is flipped 10 times? 


. What is the expected sum of the numbers that appear on 


two dice, each biased so that a 3 comes up twice as often 
as each other number? 


. What is the expected value when a $1 lottery ticket is 


bought in which the purchaser wins exactly $10 million 
if the ticket contains the six winning numbers chosen from 


the set {1, 2,3, ..., 50} and the purchaser wins nothing 
otherwise? 
. The final exam of a discrete mathematics course con- 


sists of 50 true/false questions, each worth two points, 
and 25 multiple-choice questions, each worth four points. 
The probability that Linda answers a true/false question 
correctly is 0.9, and the probability that she answers a 
multiple-choice question correctly is 0.8. What is her ex- 
pected score on the final? 


. What is the expected sum of the numbers that appear when 


three fair dice are rolled? 


. Suppose that the probability that x is in a list of n distinct 


integers is 2/3 and that it is equally likely that x equals 
any element in the list. Find the average number of com- 
parisons used by the linear search algorithm to find x or 
to determine that it is not in the list. 

Suppose that we flip a fair coin until either it comes up 
tails twice or we have flipped it six times. What is the 
expected number of times we flip the coin? 

Suppose that we roll a fair die until a 6 comes up or we 
have rolled it 10 times. What is the expected number of 
times we roll the die? 

Suppose that we roll a fair die until a 6 comes up. 

a) What is the probability that we roll the die n times? 
b) What is the expected number of times we roll the die? 
Suppose that we roll a pair of fair dice until the sum of 
the numbers on the dice is seven. What is the expected 
number of times we roll the dice? 


Show that the sum of the probabilities of a random vari- 
able with geometric distribution with parameter p, where 
0 < p < 1, equals 1. 

Show that if the random variable X has the geometric 
distribution with parameter p, and j is a positive integer, 
then p(X > j) = (1— pil. 

Let X and Y be the random variables that count the num- 
ber of heads and the number of tails that come up when 
two fair coins are flipped. Show that X and Y are not 
independent. 


17. Estimate the expected number of integers with 1000 dig- 
its that need to be selected at random to find a prime, 
if the probability a number with 1000 digits is prime is 
approximately 1/2302. 


18. Suppose that X and Y are random variables and that 
X and Y are nonnegative for all points in a sample 
space S. Let Z be the random variable defined by 
Z(s) = max(X(s), Y(s)) for all elements s € S. Show 
that E(Z) < E(X)+ E(Y). 

19. Let X be the number appearing on the first die when two 
fair dice are rolled and let Y be the sum of the numbers ap- 
pearing on the two dice. Show that E(X) E(Y) 4 E(XY). 


* 20. Show that if X,, X2,..., X, are mutually independent 
random variables, then E([]/_, Xi) = [J], E(Xi). 
The conditional expectation of the random variable X 
given the event A from the sample space S is E(X|A) = 

rex(sy” - P(X =r|A). 

21. What is expected value of the sum of the numbers ap- 
pearing on two fair dice when they are rolled given that 
the sum of these numbers is at least nine. That is, what 
is E(X|A) where X is the sum of the numbers appearing 
on the two dice and A is the event that X > 9? 

The law of total expectation states that if the sample 

space § is the disjoint union of the events S;, S2,..., Sy 

and xX is a random variable, then EF(X)= 

Vin E(X1S;)P(Sj). 

22. Prove the law of total expectations. 


23. Use the law of total expectation to find the average weight 
of a breeding elephant seal, given that 12% of the breed- 
ing elephant seals are male and the rest are female, and 
the expected weights of a breeding elephant seal is 4,200 
pounds for a male and 1,100 pounds for a female. 


24. Let A be an event. Then /,, the indicator random 
variable of A, equals | if A occurs and equals 0 oth- 
erwise. Show that the expectation of the indicator ran- 
dom variable of A equals the probability of A, that is, 
E(Ia) = pA). 

25. A run is a maximal sequence of successes in a se- 
quence of Bernoulli trials. For example, in the sequence 
S,S,S,F,S,S, F, F, S, where S represents success and 
F represents failure, there are three runs consisting of 
three successes, two successes, and one success, respec- 
tively. Let R denote the random variable on the set of se- 
quences of n independent Bernoulli trials that counts the 
number of runs in this sequence. Find F(R). [Hint: Show 
that R = St Tj, where J; = 1 ifarun begins at the jth 
Bernoulli trial and 7; = 0 otherwise. Find E(/;) and then 
find E(I;), where 1 < j <n.] 

26. Let X(s) be a random variable, where X (s) is a nonneg- 
ative integer for all s € S, and let Ax be the event that 
X(s) = k. Show that E(X) = 7Po, p(Ax). 

27. What is the variance of the number of heads that come up 
when a fair coin is flipped 10 times? 


28. 


29. 


30. 


31. 


32. 


33. 


* 34, 


35. 


36. 


37. 


38. 


What is the variance of the number of times a 6 appears 

when a fair die is rolled 10 times? 

Let X, be the random variable that equals the number 

of tails minus the number of heads when n fair coins are 

flipped. 

a) What is the expected value of X,,? 

b) What is the variance of X,,? 

Show that if X and Y are independent random 

variables, then V(XY) = E(X)?V(Y) + E(Y)?V(X) + 

ViXQyvyy) 

Let A(X) = E(|X — E(X)|), the expected value of the 

absolute value of the deviation of X, where X is a 

random variable. Prove or disprove that A(X + Y) = 

A(X) + A(Y) for all random variables X and Y. 

Provide an example that shows that the variance of the 

sum of two random variables is not necessarily equal to 

the sum of their variances when the random variables are 
not independent. 

Suppose that X, and X» are independent Bernoulli 

trials each with probability 1/2, and let X3= 

(X; + X2)mod 2. 

a) Show that X1, X2, and X3 are pairwise independent, 
but X3 and X; + X2 are not independent. 

b) Show that V(X; + X2 + X3) = V(X1) + V(X2) + 
V(X3). 

c) Explain why a proof by mathematical induction of 
Theorem 7 does not work by considering the random 
variables X;, X2, and X3. 

Prove the general case of Theorem 7. That is, show that 

if X1, X2,...,Xn are pairwise independent ran- 

dom variables on a sample space S, where n is 

a positive integer, then V(X, + X24+---+X,) = 

V(X1) + V(X2) +--+ + V(Xn). [Hint: Generalize the 

proof given in Theorem 7 for two random variables. Note 

that a proof using mathematical induction does not work; 

see Exercise 33.] 

Use Chebyshev’s inequality to find an upper bound on the 

probability that the number of tails that come up when a 

fair coin is tossed n times deviates from the mean by more 

than 5./n. 

Use Chebyshev’s inequality to find an upper bound on the 

probability that the number of tails that come up when a 

biased coin with probability of heads equal to 0.6 is tossed 

n times deviates from the mean by more than ./n. 

Let X be a random variable on a sample space 

S such that X(s)>0O for all s eS. Show that 

p(X(s) = a) < E(X)/a for every positive real num- 

ber a. This inequality is called Markov’s inequality. 

Suppose that the number of cans of soda pop filled in a day 

at a bottling plant is a random variable with an expected 

value of 10,000 and a variance of 1000. 


a) Use Markov’s inequality (Exercise 37) to obtain an 
upper bound on the probability that the plant will fill 
more than 11,000 cans on a particular day. 

b) Use Chebyshev’s inequality to obtain a lower bound 
on the probability that the plant will fill between 9000 
and 11,000 cans on a particular day. 


39. 


* 40. 


*4I, 


* 42, 
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Suppose that the number of tin cans recycled in a day at 
a recycling center is a random variable with an expected 
value of 50,000 and a variance of 10,000. 


a) Use Markov’s inequality (Exercise 37) to find an up- 
per bound on the probability that the center will recy- 
cle more than 55,000 cans on a particular day. 


b) Use Chebyshev’s inequality to provide a lower bound 
on the probability that the center will recycle 40,000 
to 60,000 cans on a certain day. 


Suppose the probability that x is the ith element in a list 
of n distinct integers is i/[m(m + 1)]. Find the average 
number of comparisons used by the linear search algo- 
rithm to find x or to determine that it is not in the list. 


In this exercise we derive an estimate of the average-case 
complexity of the variant of the bubble sort algorithm 
that terminates once a pass has been made with no inter- 
changes. Let X be the random variable on the set of per- 
mutations of a set of n distinct integers {a,, a2,..., dy} 
witha, < az <--- < a, suchthat X (P) equals the num- 
ber of comparisons used by the bubble sort to put these 
integers into increasing order. 

a) Show that, under the assumption that the input is 
equally likely to be any of the n! permutations of these 
integers, the average number of comparisons used by 
the bubble sort equals F(X). 


b) Use Example 5 in Section 3.3 to show that E(X) < 
n(n — 1)/2. 


c) Show that the sort makes at least one comparison for 
every inversion of two integers in the input. 


d) Let J(P) be the random variable that equals the num- 
ber of inversions in the permutation P. Show that 
E(X) > Ed). 


e) Let J;,, be the random variable with [;,,(P) = 1 if ax 
precedes a; in P and I; , = 0 otherwise. Show that 


T(P) = Ve Vj ck Lx (P). 
f) Show that EI) = Yj, E(1j.4)- 


g) Show that E(/;,~) = 1/2. [Hint: Show that E(Jj,.) = 
probability that a, precedes aj; in a permutation P. 
Then show it is equally likely for a, to precede a; as 
it is for aj to precede a,x in a permutation.] 


h) Use parts (f) and (g) to show that E(/) = n(n — 1)/4. 


i) Conclude from parts (b), (d), and (h) that the aver- 
age number of comparisons used to sort n integers is 
O(n’). 

In this exercise we find the average-case complexity of 

the quick sort algorithm, described in the preamble to Ex- 

ercise 50 in Section 5.4, assuming a uniform distribution 
on the set of permutations. 

a) Let X be the number of comparisons used by the quick 
sort algorithm to sort a list of n distinct integers. Show 
that the average number of comparisons used by the 
quick sort algorithm is E (X) (where the sample space 
is the set of all n! permutations of n integers). 
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b) Let 7; denote the random variable that equals 1 if 
the jth smallest element and the kth smallest element 
of the initial list are ever compared as the quick sort 
algorithm sorts the list and equals 0 otherwise. Show 
that X = Dytg ja Tj 

c) Show that E(X) = }°7_, pam p(the jth smallest 
element and the kth smallest element are compared). 

d) Show that p(the jth smallest element and the kth 
smallest element are compared), where k > j, equals 
2/(k—j+1). 

e) Use parts (c) and (d) to show that E(X)= 
2(n + 1), 1/i) — 2(n - 1). 

f) Conclude from part (e) and the fact that ce 1/j ~ 
Inn + y, where y = 0.57721... is Euler’s constant, 
that the average number of comparisons used by the 
quick sort algorithm is O(n logn). 

* 43, What is the variance of the number of fixed elements, 
that is, elements left in the same position, of a randomly 
selected permutation of n elements? [Hint: Let X denote 
the number of fixed points of arandom permutation. Write 
X= X,+X24+---+ X,, where X; = 1 if the permu- 
tation fixes the ith element and X; = 0 otherwise.] 


Key Terms and Results 


TERMS 
sample space: the set of possible outcomes of an experiment 
event: a subset of the sample space of an experiment 


probability of an event (Laplace’s definition): the number 
of successful outcomes of this event divided by the number 
of possible outcomes 


probability distribution: a function p from the set of all out- 
comes of a sample space § for which 0 < p(x;) < 1 for 
i=1,2,...,n and ~"_, p(x) = 1, where xj,..., x, are 
the possible outcomes 


probability of an event E: the sum of the probabilities of the 
outcomes in E 


P(E|F) (conditional probability of E given F): the ratio 
D(EN F)/p(F) 

independent events: events E and F such that p(EM F) = 
D(E)p(F) 

pairwise independent events: events E,, E2,..., En, such 
that p(E; N Ej) = p(E;)p(E;) for all pairs of integers i 
and j wih|l1<j<k<n 

mutually independent events: events EF), F2,..., E, such 
that p( Ei, 1 Big A+++ E;,,) = p(Ei,) p(Bin) +++ P(Ei,,) 
whenever i;, j = 1,2,...,m, are integers with | < ij < 
ig <-+++<im <nandm>2 


random variable: a function that assigns a real number to each 
possible outcome of an experiment 


The covariance of two random variables X and Y ona sample 

space S, denoted by Cov(X, Y), is defined to be the expected 

value of the random variable (X — E(X))(Y — E(Y)). That 

is, Cov(X, Y) = E((X — E(X))(Y — E(Y))). 

44. Show that Cov(X, Y) = E(XY) — E(X)E(Y), and use 
this result to conclude that Cov(X, Y) = Oif X and Y are 
independent random variables. 


45. Show that V(X + Y) = V(X) 4+ V(Y) + 2 Cov(X, Y). 


46. Find Cov(X, Y) if X and Y are the random variables with 
X(G, j)) = 2i and Y((i, j)) =i + j, where i and j are 
the numbers that appear on the first and second of two 
dice when they are rolled. 


47. When m balls are distributed into n bins uniformly at 
random, what is the probability that the first bin remains 
empty? 

48. What is the expected number of balls that fall into the first 


bin when m balls are distributed into n bins uniformly at 
random? 


49. What is the expected number of bins that remain empty 
when m balls are distributed into n bins uniformly at ran- 
dom? 


distribution of a random variable X: the set of pairs 
(r, p(X =7r)) forr € X(S) 

uniform distribution: the assignment of equal probabilities 
to the elements of a finite set 

expected value of a random variable: the weighted av- 
erage of a random variable, with values of the random 
variable weighted by the probability of outcomes, that is, 
E(X) = Yises PS) X (5) 

geometric distribution: the distribution of a random variable 
X such that p(X =k) = (1— p)‘'p fork = 1, 2,... for 
some real number p with O < p < 1. 

independent random variables: random variables X and Y 
such that p(X =r, and Y = rz) = p(X =1r1)p(Y =r) 
for all real numbers r; and rz 


variance of a random variable X : the weighted average of the 
square of the difference between the value of X and its ex- 
pected value E(X), with weights given by the probability 
of outcomes, that is, V(X) = Y\ye5(X(s) — E(X))? p(s) 

standard deviation of a random variable X: the square root 
of the variance of X, that is, o(X) = / V(X) 

Bernoulli trial: an experiment with two possible outcomes 

probabilistic (or Monte Carlo) algorithm: an algorithm in 
which random choices are made at one or more steps 

probabilistic method: a technique for proving the existence 
of objects in a set with certain properties that proceeds by 
assigning probabilities to objects and showing that the prob- 
ability that an object has these properties is positive 


RESULTS 


The probability of exactly k successes when n indepen- 
dent Bernoulli trials are carried out equals C(n, k) p*g"—*, 
where p is the probability of success and g = 1 — p is the 
probability of failure. 

Bayes’ theorem: If FE and F are events from a sample space 
S such that p(E) 4 0 and p(F) 4 0, then 


= p(E | F)p(F) 
p(E | F)p(F) + p(E | F)p(F) 


E(X) = Vrexcsy POX = r)r. 


P(F | E) 


Review Questions 


1. a) Define the probability of an event when all outcomes 
are equally likely. 
b) What is the probability that you select the six winning 
numbers in a lottery if the six different winning num- 
bers are selected from the first 50 positive integers? 


2. a) What conditions should be met by the probabilities 
assigned to the outcomes from a finite sample space? 
b) What probabilities should be assigned to the outcome 
of heads and the outcome of tails if heads comes up 
three times as often as tails? 


3. a) Define the conditional probability of an event E given 
an event F. 

b) Suppose E is the event that when a die is rolled it 
comes up an even number, and F is the event that 
when a die is rolled it comes up 1, 2, or 3. What is the 
probability of F given EF? 

4. a) When are two events EF and F independent? 

b) Suppose E is the event that an even number appears 
when a fair die is rolled, and F is the event that a 5 
or 6 comes up. Are E and F independent? 


5. a) What is a random variable? 
b) What are the possible values assigned by the random 
variable X that assigns to a roll of two dice the larger 
number that appears on the two dice? 


6. a) Define the expected value of a random variable X. 
b) What is the expected value of the random variable X 
that assigns to a roll of two dice the larger number that 
appears on the two dice? 


7. a) Explain how the average-case computational com- 
plexity of an algorithm, with finitely many possible 
input values, can be interpreted as an expected value. 

b) What is the average-case computational complexity 
of the linear search algorithm, if the probability that 
the element for which we search is in the list is 1/3, 
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linearity of expectations: F(X; + X2+---+ X,)=E(X1) + 
E(X2)+---+ E(X,) if X1, Xo, ..., X, are random vari- 
ables 


If X and Y are independent random variables, then E(XY) = 
E(X)E(Y). 

Bienaymé’s formula: If X;, Xo, ..., X, are independent ran- 
dom variables, then V(X; + X2 +---+ X,) = V(X1) + 
V(X2) +--+ + V (Xn). 

Chebyshev’s inequality: p(|X (s) — E(X)| => r) < V(X)/r?, 
where X is a random variable with probability function p 
and r is a positive real number. 


and it is equally likely that this element is any of 
the n elements in the list? 


8. a) What is meant by a Bernoulli trial? 
b) What is the probability of k successes inn independent 
Bernoulli trials? 
c) What is the expected value of the number of successes 
in n independent Bernoulli trials? 
9. a) What does the linearity of expectations of random vari- 
ables mean? 
b) How can the linearity of expectations help us find the 
expected number of people who receive the correct 
hat when a hatcheck person returns hats at random? 


10. a) How can probability be used to solve a decision prob- 

lem, if a small probability of error is acceptable? 

b) How can we quickly determine whether a positive in- 
teger is prime, if we are willing to accept a small prob- 
ability of making an error? 

11. State Bayes’ theorem and use it to find p(F | E) if 
P(E| F)=1/3, p(E| F)=1/4, and p(F) = 2/3, 
where E and F are events from a sample space S. 

12. a) What does it mean to say that a random variable has a 

geometric distribution with parameter p? 

b) What is the mean of a geometric distribution with pa- 
rameter p? 

13. a) What is the variance of a random variable? 

b) What is the variance of a Bernoulli trial with proba- 
bility p of success? 

14. a) What is the variance of the sum of n independent ran- 

dom variables? 

b) What is the variance of the number of successes when 
n independent Bernoulli trials, each with probability p 
of success, are carried out? 

15. What does Chebyshev’s inequality tell us about the prob- 
ability that a random variable deviates from its mean by 
more than a specified amount? 
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Supplementary Exercises 


1. What is the probability that six consecutive integers will c) pairs of each of three different kinds and a single card 


be chosen as the winning numbers in a lottery where each 
number chosen is an integer between | and 40 (inclusive)? 


2. A player in the Mega Millions lottery picks five different 


integers between 1 and 56, inclusive, and a sixth integer 

between | and 46, which may duplicate one of the earlier 

five integers. The player wins the jackpot if the first five 
numbers picked match the first five numbers drawn and 
the sixth number matches the sixth number drawn. 

a) What is the probability that a player wins the jackpot? 

b) What is the probability that a player wins $250,000, 
which is the prize for matching the first five numbers, 
but not the sixth number, drawn? 

c) What is the probability that a player wins $150 by 
matching exactly three of the first five numbers and 
the sixth number or by matching four of the first five 
numbers but not the sixth number? 

d) What is the probability that a player wins a prize, if a 
prize is given when the player matches at least three 
of the first five numbers or the last number. 


3. A player in the Powerball lottery picks five different in- 


tegers between | and 59, inclusive, and a sixth integer 

between | and 39, which may duplicate one of the earlier 

five integers. The player wins the jackpot if the first five 
numbers picked match the first five number drawn and 
the sixth number matches the sixth number drawn. 

a) What is the probability that a player wins the jackpot? 

b) What is the probability that a player wins $200,000, 
which is the prize for matching the first five numbers, 
but not the sixth number, drawn? 

c) What is the probability that a player wins $100 by 
matching exactly three of the first five and the sixth 
numbers or four of the first five numbers but not the 
sixth number? 

d) What is the probability that a player wins a prize, if a 
prize is given when the player matches at least three 
of the first five numbers or the last number. 


4. What is the probability that a hand of 13 cards contains 


no pairs? 


5. What is the probability that a 13-card bridge hand contains 


a) all 13 hearts? 

b) 13 cards of the same suit? 

c) seven spades and six clubs? 

d) seven cards of one suit and six cards of a second suit? 
e) four diamonds, six hearts, two spades, and one club? 


f) four cards of one suit, six cards of a second suit, two 
cards of a third suit, and one card of the fourth suit? 


6. What is the probability that a seven-card poker hand con- 


tains 

a) four cards of one kind and three cards of a second 
kind? 

b) three cards of one kind and pairs of each of two dif- 
ferent kinds? 


of a fourth kind? 


d) pairs of each of two different kinds and three cards of 
a third, fourth, and fifth kind? 


e) cards of seven different kinds? 
f) aseven-card flush? 

g) aseven-card straight? 

h) aseven-card straight flush? 


An octahedral die has eight faces that are numbered 1 
through 8. 


7. 


a) What is the expected value of the number that comes 
up when a fair octahedral die is rolled? 

b) What is the variance of the number that comes up when 
a fair octahedral die is rolled? 


A dodecahedral die has 12 faces that are numbered 1 
through 12. 


8. 


10. 


11. 


12. 


13. 


a) What is the expected value of the number that comes 
up when a fair dodecahedral die is rolled? 

b) What is the variance of the number that comes up when 
a fair dodecahedral die is rolled? 


. Suppose that a pair of fair octahedral dice is rolled. 


a) What is the expected value of the sum of the numbers 
that come up? 

b) What is the variance of the sum of the numbers that 
come up? 

Suppose that a pair of fair dodecahedral dice is rolled. 

a) What is the expected value of the sum of the numbers 
that come up? 

b) What is the variance of the sum of the numbers that 
come up? 

Suppose that a fair standard (cubic) die and a fair octahe- 

dral die are rolled together. 

a) What is the expected value of the sum of the numbers 
that come up? 

b) What is the variance of the sum of the numbers that 
come up? 

Suppose that a fair octahedral die and a fair dodecahedral 

die are rolled together. 

a) What is the expected value of the sum of the numbers 
that come up? 

b) What is the variance of the sum of the numbers that 
come up? 

Suppose n people, n > 3, play “odd person out” to de- 

cide who will buy the next round of refreshments. The n 

people each flip a fair coin simultaneously. If all the coins 

but one come up the same, the person whose coin comes 

up different buys the refreshments. Otherwise, the people 

flip the coins again and continue until just one coin comes 

up different from all the others. 

a) What is the probability that the odd person out is de- 
cided in just one coin flip? 


14. 


*15, 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


b) What is the probability that the odd person out is de- 
cided with the kth flip? 

c) What is the expected number of flips needed to decide 
odd person out with n people? 

Suppose that p and q are primes andn = pq. What is the 

probability that a randomly chosen positive integer less 

than n is not divisible by either p or g? 


Suppose that m and n are positive integers. What is the 
probability that a randomly chosen positive integer less 
than mn is not divisible by either m or n? 


Suppose that E,, E2,..., E, aren events with p(E;) > 0 
fori = 1,2,...,n. Show that 


P(E N E2N--+-M En) 
= p(E)) p(E2 | Ei) p(E3 | £1 9 E2) 
++ D(En | FE, FE. 9-+--0 En-1). 


There are three cards in a box. Both sides of one card are 

black, both sides of one card are red, and the third card 

has one black side and one red side. We pick a card at 

random and observe only one side. 

a) If the side is black, what is the probability that the 
other side is also black? 

b) What is the probability that the opposite side is the 
same color as the one we observed? 


What is the probability that when a fair coin is flipped n 
times an equal number of heads and tails appear? 


What is the probability that a randomly selected bit string 
of length 10 is a palindrome? 


What is the probability that a randomly selected bit string 
of length 11 is a palindrome? 


Consider the following game. A person flips a coin re- 
peatedly until a head comes up. This person receives a 
payment of 2” dollars if the first head comes up at the 
nth flip. 

a) Let X be a random variable equal to the amount of 
money the person wins. Show that the expected value 
of X does not exist (that is, it is infinite). Show that 
a rational gambler, that is, someone willing to pay to 
play the game as long as the price to play is not more 
than the expected payoff, should be willing to wager 
any amount of money to play this game. (This is known 
as the St. Petersburg paradox. Why do you suppose 
it is called a paradox?) 

Suppose that the person receives 2” dollars if the 
first head comes up on the nth flip where n < 8 and 
28 = 256 dollars if the first head comes up on or af- 
ter the eighth flip. What is the expected value of the 
amount of money the person wins? How much money 
should a person be willing to pay to play this game? 
Suppose that n balls are tossed into b bins so that each 
ball is equally likely to fall into any of the bins and that 
the tosses are independent. 


b 


hue 


a) Find the probability that a particular ball lands in a 
specified bin. 


23 


24. 


25. 


26. 
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b) What is the expected number of balls that land in a 
particular bin? 


c) What is the expected number of balls tossed until a 
particular bin contains a ball? 


*d) What is the expected number of balls tossed until all 
bins contain a ball? [Hint: Let X; denote the number 
of tosses required to have a ball land in an ith bin 
once i — | bins contain a ball. Find F(X; ) and use the 
linearity of expectations. ] 


. Suppose that A and B are events with probabilities 


D(A) = 3/4 and p(B) = 173. 

a) What is the largest p(AM B) can be? What is the 
smallest it can be? Give examples to show that both 
extremes for p(A /M B) are possible. 


b) What is the largest p(A U B) can be? What is the 
smallest it can be? Give examples to show that both 
extremes for p(A U B) are possible. 

Suppose that A and B are events with probabilities 

p(A) = 2/3 and p(B) = 1/2. 

a) What is the largest p(AM B) can be? What is the 
smallest it can be? Give examples to show that both 
extremes for p(A M B) are possible. 


b) What is the largest p(A U B) can be? What is the 
smallest it can be? Give examples to show that both 
extremes for p(A U B) are possible. 

Recall from Definition 5 in Section 7.2 that the 

events E,, E2,..., E, are mutually independent if 

P(Ei, Ein N---O Ei) = pl Ei) pin) +++ P(Ein) 

wheneveri;, j = 1,2,...,m, are integers with 1 <i, < 

ig <-+++ <im <nandm > 2. 

a) Write out the conditions required for three 
events FE, E2, and £3 to be mutually independent. 


b) Let £1, E2, and E3 be the events that the first flip 
comes up heads, that the second flip comes up tails, 
and that the third flip comes up tails, respectively, 
when a fair coin is flipped three times. Are FE), Fo, 
and £3 mutually independent? 


c) Let £1, E2, and E3 be the events that the first flip 
comes up heads, that the third flip comes up heads, 
and that an even number of heads come up, respec- 
tively, when a fair coin is flipped three times. Are E), 
E2, and E3 pairwise independent? Are they mutually 
independent? 


d) Let £1, E2, and E3 be the events that the first flip 
comes up heads, that the third flip comes up heads, 
and that exactly one of the first flip and third flip come 
up heads, respectively, when a fair coin is flipped three 
times. Are FE), Ez, and E3 pairwise independent? Are 
they mutually independent? 


e) How many conditions must be checked to show that n 
events are mutually independent? 

Suppose that A and B are events from a sample 

space S such that p(A) 4 0 and p(B) ¥ 0. Show that if 

p(B | A) < p(B), then p(A | B) < p(A). 
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In Exercise 27 we consider the two children problem, intro- 
duced in 1959 by Martin Garnder is his Mathematical Games 
column in Scientific American. A version of the puzzle asks: 
“We meet Mr. Smith as he is walking down the street with a 
young child whom he introduces as his son. He also tells us 
that he has two children. What is the probability that his other 
child is a son?” We will show that this puzzle is ambiguous, 
leading to a paradox, by showing that there are two reasonable 
answers to this problem and we will describe how to make the 
puzzle unambiguous. 


* 27, 


28. 


29. 


30. 


31. 


32. 


33. 


a) Solve this puzzle in two different ways. First, answer 
the problem by considering the probability of the gen- 
der of the second child. Then, determine the probabil- 
ity differently, by considering the four different possi- 
bilities for a family of two children. 

b) Show that the answer to the puzzle becomes unam- 
biguous if we also know that Mr. Smith chose his 
walking companion at random from his two children. 

c) Another variation of this puzzle asks “When we meet 
Mr. Smith, he tells us that he has two children and 
at least one is a son. What is the probability that his 
other child is ason?” Solve this variation of the puzzle, 
explaining why it is unambiguous. 

In 2010, the puzzle designer Gary Foshee posed this prob- 

lem: “Mr. Smith has two children, one of whom is a son 

born on a Tuesday. What is the probability that Mr. Smith 
has two sons?” Show that there are two different answers 
to this puzzle, depending on whether Mr. Smith specifi- 
cally mentioned his son because he was born on a Tues- 
day or whether he randomly chose a child and reported 
its gender and birth day of the week. [Hint: For the first 
possibility, enumerate all the equally likely possibilities 
for the gender and birth day of the week of the other child. 

To do, this consider first the cases where the older child 

is a boy born on a Tuesday and then the case where the 

older child is not a boy born on a Tuesday.] 


Let X be a random variable on a sample space S. Show 
that V(aX + b) = a2V(X) whenever a and b are real 
numbers. 


Use Chebyshev’s inequality to show that the probability 
that more than 10 people get the correct hat back when 
a hatcheck person returns hats at random does not ex- 
ceed 1/100 no matter how many people check their hats. 
[Hint: Use Example 6 and Exercise 43 in Section 7.4.] 


Suppose that at least one of the events Ej, j = 
1,2,...,m,1s guaranteed to occur and no more than two 
can occur. Show that if p(E;) = q for j =1,2,...,m 
and p(E; 1 Ex) =r forl < j <k <m, theng = 1/m 
andr < 2/m. 

Show that if m is a positive integer, then the probability 
that the mth success occurs on the (m + n)th trial when 
independent Bernoulli trials, each with probability p of 
success, are run, is gp 

There are n different types of collectible cards you can 
get as prizes when you buy a particular product. Suppose 
that every time you buy this product it is equally likely 
that you get any type of these cards. Let X be the random 


34. 


35. 


variable equal to the number of products that need to be 
purchased to obtain at least one of each type of card and 
let X ; be the random variable equal to the number of ad- 
ditional products that must be purchased after j different 
cards have been collected until a new card is obtained 
for 7 =0,1,...,n—1. 

a) Show that X = "ly Xj. 

b) Show that after 7 distinct types of cards have been ob- 


tained, the card obtained with the next purchase will 
be a card of a new type with probability (n — j)/n. 


c) Show that X; has a geometric distribution with pa- 
rameter (n — j)/n. 

d) Use parts (a) and (c) to show that E(X)= 
nya I/j. 

e) Use the approximation es 1/j7 + Inn+ y, where 
y = 0.57721... is Euler’s constant, to find the ex- 
pected number of products that you need to buy to get 
one card of each type if there are 50 different types of 
cards. 


The maximum satisfiability problem asks for an as- 
signment of truth values to the variables in a compound 
proposition in conjunctive normal form (which expresses 
a compound proposition as the conjunction of clauses 
where each clause is the disjunction of two or more vari- 
ables or their negations) that makes as many of these 
clauses true as possible. For example, three but not four 
of the clauses in 


(PV QA(PV 7g) 4 (AP Vr) A (pV -7r) 


can be made true by an assignment of truth values to p, q, 
and r. We will show that probabilistic methods can pro- 
vide a lower bound for the number of clauses that can be 
made true by an assignment of truth values to the vari- 
ables. 


a) Suppose that there are n variables in a compound 
proposition in conjunctive normal form. If we pick 
a truth value for each variable randomly by flipping 
a coin and assigning true to the variable if the coin 
comes up heads and false if it comes up tails, what 
is the probability of each possible assignment of truth 
values to the n variables? 


b) Assuming that each clause is the disjunction of ex- 
actly two distinct variables or their negations, what is 
the probability that a given clause is true, given the 
random assignment of truth values from part (a)? 


c) Suppose that there are D clauses in the compound 
proposition. What is the expected number of these 
clauses that are true, given the random assignment of 
truth values of the variables? 

d) Use part (c) to show that for every compound proposi- 
tion in conjunctive normal form there is an assignment 
of truth values to the variables that makes at least 3/4 
of the clauses true. 


What is the probability that each player has a hand con- 
taining an ace when the 52 cards of a standard deck are 
dealt to four players? 


* 36. The following method can be used to generate a ran- 


dom permutation of a sequence of n terms. First, in- 
terchange the nth term and the r(n)th term where r(n) 
is a randomly selected integer with 1 < r(m) < n. 
Next, interchange the (m —1)st term of the result- 
ing sequence with its r(m — 1)st term where r(n — 1) 
is a randomly selected integer with 1<r(n—1)< 
n—1. Continue this process until j =n, where at 
the jth step you interchange the (n — j + 1)st term 


Computer Projects 


Write programs with these input and output. 


. Given areal number p with0O < p < 1, generate random 
numbers taken from a Bernoulli distribution with proba- 
bility p. 

. Given a positive integer n, generate a random permuta- 
tion of the set {1,2,3,...,n}. (See Exercise 36 in the 
Supplementary Exercises.) 


. Given positive integers m and n, generate m random per- 
mutations of the first n positive integers. Find the number 
of inversions in each permutation and determine the av- 
erage number of these inversions. 


. Given a positive integer n, simulate n repeated flips of 
a biased coin with probability p of heads and determine 
the number of heads that come up. Display the cumulative 
results. 


. Given positive integers n and m, generate m random per- 
mutations of the first n positive integers. Sort each per- 
mutation using the insertion sort, counting the number 
of comparisons used. Determine the average number of 
comparisons used over all m permutations. 

. Given positive integers n and m, generate m random per- 
mutations of the first n positive integers. Sort each permu- 
tation using the version of the bubble sort that terminates 


10. 


11. 
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of the resulting sequence with its r(n — j + 1)st term, 
where r(n — j7 + 1) is a randomly selected integer with 
l<ra—j+1)<n-—j+1. Show that when this 
method is followed, each of the n! different permutations 
of the terms of the sequence is equally likely to be gen- 
erated. [Hint: Use mathematical induction, assuming that 
the probability that each of the permutations of n — 1 
terms produced by this procedure for a sequence of n — 1 
terms is 1/(n — 1)!.] 


when a pass has been made with no interchanges, counting 
the number of comparisons used. Determine the average 
number of comparisons used over all m permutations. 


. Given a positive integer m, simulate the collection of cards 


that come with the purchase of products to find the num- 
ber of products that must be purchased to obtain a full 
set of m different collector cards. (See Supplementary 
Exercise 33.) 


. Given positive integers m and n, simulate the placement 


of n keys, where a record with key k is placed at location 
h(k) = k mod m and determine whether there is at least 
one collision. 


. Given a positive integer n, find the probability of select- 


ing the six integers from the set {1,2,...,m} that were 
mechanically selected in a lottery. 

Simulate repeated trials of the Monty Hall Three-Door 
problem (Example 10 in Section 7.1) to calculate the prob- 
ability of winning with each strategy. 

Given a list of words and the empirical probabilities they 
occur in spam e-mails and in e-mails that are not spam, 
determine the probability that a new e-mail message is 
spam. 


Computations and Explorations 


Use a computational program or programs you have written to do these exercises. 


1. Find the probabilities of each type of hand in five-card 3. Estimate the probability that two integers selected at ran- 


poker and rank the types of hands by their probability. 


. Find some conditions such that the expected value of buy- 
ing a $1 lottery ticket in the New Jersey Pick-6 lottery has 
an expected value of more than $1. To win you have to se- 
lect the six numbers drawn, where order does not matter, 
from the positive integers | to 49, inclusive. The winnings 
are split evenly among holders of winning tickets. Be sure 
to consider the total size of the pot going into the drawing 
and the number of people buying tickets. 


dom are relatively prime by testing a large number of 
randomly selected pairs of integers. Look up the theorem 
that gives this probability and compare your results with 
the correct probability. 


. Determine the number of people needed to ensure that the 


probability at least two of them have the same day of the 
year as their birthday is at least 70%, at least 80%, at least 
90%, at least 95%, at least 98%, and at least 99%. 


500 7 / Discrete Probability 


5. Generate a list of 100 randomly selected permutations of 
the set of the first 100 positive integers. (See Exercise 36 
in the Supplementary Exercises.) 

6. Given acollection of e-mail messages, each determined to 
be spam or not to be spam, develop a Bayesian filter based 
on the appearance of particular words in these messages. 

7. Simulate the odd-person-out procedure (described in Ex- 
ercise 13 of the Supplementary Exercises) for n peo- 
ple with 3 < n < 10. Run a large number of trials for 


Writing Projects 


Respond to these with essays using outside sources. 


. Describe the origins of probability theory and the first uses 
of this theory, including those by Cardano, Pascal, and 
Laplace. 

. Describe the different bets you can make when you 
play roulette. Find the probability of each of these bets 
in the American version where the wheel contains the 
numbers 0 and 00. Which is the best bet and which is the 
worst for you? 

. Discuss the probability of winning when you play the game 
of blackjack versus a casino. Is there a winning strategy for 
the person playing against the house? 

. Investigate the game of craps and discuss the probability 
that the shooter wins and how close to a fair game it is. 


each value of n and use the results to estimate the ex- 
pected number of flips needed to find the odd person out. 
Does your result agree with that found in Exercise 29 in 
Section 7.2? Vary the problem by supposing that exactly 
one person has a biased coin with probability of heads 
p #05. 


. Given a positive integer n, simulate a hatcheck person ran- 


domly giving hats back to people. Determine the number 
of people who get the correct hat back. 


. Discuss issues involved in developing successful spam fil- 


ters and the current situation in the war between spammers 
and people trying to filter spam out. 


. Discuss the history and solution of what is known as 


the Newton—Pepys problem, which asks which is most 
likely: rolling at least one six when six dice are rolled, 
rolling at least two sixes when 12 dice are rolled, or rolling 
at least three sixes when 18 dice are rolled. 


. Explain how Erdés and Rényi first used the probabilis- 


tic method and describe some other applications of this 
method. 


. Discuss the different types of probabilistic algorithms and 


describe some examples of each type. 
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Advanced Counting Techniques 


M any counting problems cannot be solved easily using the methods discussed in Chapter 6. 
One such problem is: How many bit strings of length n do not contain two consecutive 
zeros? To solve this problem, let a, be the number of such strings of length n. An argument can 
be given that shows that the sequence {a,} satisfies the recurrence relation dy)+1 = dy + dn—1 
and the initial conditions aj = 2 and az = 3. This recurrence relation and the initial conditions 
determine the sequence {a,, }. Moreover, an explicit formula can be found for a, from the equation 
relating the terms of the sequence. As we will see, a similar technique can be used to solve many 
different types of counting problems. 

We will discuss two ways that recurrence relations play important roles in the study of 
algorithms. First, we will introduce an important algorithmic paradigm known as dynamic 
programming. Algorithms that follow this paradigm break down a problem into overlapping 
subproblems. The solution to the problem is then found from the solutions to the subproblems 
through the use of a recurrence relation. Second, we will study another important algorithmic 
paradigm, divide-and-conquer. Algorithms that follow this paradigm can be used to solve a 
problem by recursively breaking it into a fixed number of nonoverlapping subproblems until 
these problems can be solved directly. The complexity of such algorithms can be analyzed using 
a special type of recurrence relation. In this chapter we will discuss a variety of divide-and- 
conquer algorithms and analyze their complexity using recurrence relations. 

We will also see that many counting problems can be solved using formal power series, 
called generating functions, where the coefficients of powers of x represent terms of the sequence 
we are interested in. Besides solving counting problems, we will also be able to use generating 
functions to solve recurrence relations and to prove combinatorial identities. 

Many other kinds of counting problems cannot be solved using the techniques discussed in 
Chapter 6, such as: How many ways are there to assign seven jobs to three employees so that 
each employee is assigned at least one job? How many primes are there less than 1000? Both 
of these problems can be solved by counting the number of elements in the union of sets. We 
will develop a technique, called the principle of inclusion—exclusion, that counts the number of 
elements in a union of sets, and we will show how this principle can be used to solve counting 
problems. 

The techniques studied in this chapter, together with the basic techniques of Chapter 6, can 
be used to solve many counting problems. 


Applications of Recurrence Relations 


Introduction 


Recall from Chapter 2 that a recursive definition of a sequence specifies one or more initial terms 
and a rule for determining subsequent terms from those that precede them. Also, recall that a 
tule of the latter sort (whether or not it is part of a recursive definition) is called a recurrence 
relation and that a sequence is called a solution of a recurrence relation if its terms satisfy the 
recurrence relation. 

In this section we will show that such relations can be used to study and to solve counting 
problems. For example, suppose that the number of bacteria in a colony doubles every hour. If 
a colony begins with five bacteria, how many will be present in 7 hours? To solve this problem, 
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(at least two months old) 


let a, be the number of bacteria at the end of n hours. Because the number of bacteria doubles 
every hour, the relationship a, = 2a, —, holds whenever n is a positive integer. This recurrence 
relation, together with the initial condition ag = 5, uniquely determines a, for all nonnegative 
integers n. We can find a formula for a, using the iterative approach followed in Chapter 2, 
namely that a, = 5 - 2” for all nonnegative integers n. 

Some of the counting problems that cannot be solved using the techniques discussed in 
Chapter 6 can be solved by finding recurrence relations involving the terms of a sequence, as 
was done in the problem involving bacteria. In this section we will study a variety of counting 
problems that can be modeled using recurrence relations. In Chapter 2 we developed methods 
for solving certain recurrence relation. In Section 8.2 we will study methods for finding explicit 
formulae for the terms of sequences that satisfy certain types of recurrence relations. 

We conclude this section by introducing the algorithmic paradigm of dynamic programming. 
After explaining how this paradigm works, we will illustrate its use with an example. 


Modeling With Recurrence Relations 


We can use recurrence relations to model a wide variety of problems, such as finding compound 
interest (see Example 11 in Section2.4), counting rabbits on an island, determining the number 
of moves in the Tower of Hanoi puzzle, and counting bit strings with certain properties. 


Example | shows how the population of rabbits on an island can be modeled using a 
recurrence relation. 


Rabbits and the Fibonacci Numbers Consider this problem, which was originally posed by 
Leonardo Pisano, also known as Fibonacci, in the thirteenth century in his book Liber abaci. A 
young pair of rabbits (one of each sex) is placed on an island. A pair of rabbits does not breed 
until they are 2 months old. After they are 2 months old, each pair of rabbits produces another 
pair each month, as shown in Figure 1. Find a recurrence relation for the number of pairs of 
rabbits on the island after n months, assuming that no rabbits ever die. 


Young pairs Reproducing | Young | Total 
(less than two months old) Month pairs pairs pairs 


& A 1 0 1 1 


aa 2 0 1 1 


ana 


aa 3 1 1 2 


aa 


aa GS 4 1 2 3 
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FIGURE 1 Rabbits on an Island. 


The Fibonacci numbers 
appear in many other 
places in nature, including 
the number of petals on 
flowers and the number of 
spirals on seedheads. 


Demo 


EXAMPLE 2 


Links 


Schemes for efficiently 
backing up computer files 
on multiple tapes or other 
media are based on the 
moves used to solve the 
Tower of Hanoi puzzle. 
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Solution: Denote by f, the number of pairs of rabbits after 7 months. We will show that fy, 
n= 1,2,3,..., are the terms of the Fibonacci sequence. 

The rabbit population can be modeled using a recurrence relation. At the end of the first 
month, the number of pairs of rabbits on the island is f; = 1. Because this pair does not 
breed during the second month, /2 = | also. To find the number of pairs after n months, add 
the number on the island the previous month, f,,;, and the number of newborn pairs, which 
equals f,—2, because each newborn pair comes from a pair at least 2 months old. 

Consequently, the sequence { f,,} satisfies the recurrence relation 


tn = Sn-1 + Sn—-2 


forn > 3 together with the initial conditions f; = 1 and f2 = 1. Because this recurrence relation 
and the initial conditions uniquely determine this sequence, the number of pairs of rabbits on 
the island after n months is given by the nth Fibonacci number. < 


Example 2 involves a famous puzzle. 


The Tower of Hanoi A popular puzzle of the late nineteenth century invented by the French 
mathematician Edouard Lucas, called the Tower of Hanoi, consists of three pegs mounted on 
a board together with disks of different sizes. Initially these disks are placed on the first peg 
in order of size, with the largest on the bottom (as shown in Figure 2). The rules of the puzzle 
allow disks to be moved one at a time from one peg to another as long as a disk is never placed 
on top of a smaller disk. The goal of the puzzle is to have all the disks on the second peg in 
order of size, with the largest on the bottom. 

Let H,, denote the number of moves needed to solve the Tower of Hanoi problem with n 
disks. Set up a recurrence relation for the sequence { Hy}. 


Solution: Begin with n disks on peg 1. We can transfer the top n — | disks, following the rules 
of the puzzle, to peg 3 using H,_; moves (see Figure 3 for an illustration of the pegs and disks 
at this point). We keep the largest disk fixed during these moves. Then, we use one move to 
transfer the largest disk to the second peg. We can transfer the n — | disks on peg 3 to peg 2 
using H,,_; additional moves, placing them on top of the largest disk, which always stays fixed 
on the bottom of peg 2. Moreover, it is easy to see that the puzzle cannot be solved using fewer 
steps. This shows that 


Ay = 2Ay-1 +1. 


The initial condition is H; = 1, because one disk can be transferred from peg | to peg 2, 
according to the rules of the puzzle, in one move. 


Peg 1 


Peg 2 Peg 3 


FIGURE 2 The Initial Position in the Tower of Hanoi. 
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Peg 1 Peg 2 Peg 3 


FIGURE 3 _ An Intermediate Position in the Tower of Hanoi. 


We can use an iterative approach to solve this recurrence relation. Note that 


An = 2An-1 +1 
= 2(2H,-2+1)+1=2?Ay2+241 
= 2 QU 28 +1)+24+1= 2 Hn + 2 +241 


= 9-1 ee a Oa 
—2P-lagn-2a 4 ga] 
= 2], 


We have used the recurrence relation repeatedly to express H,, in terms of previous terms of 
the sequence. In the next to last equality, the initial condition H; = 1 has been used. The last 
equality is based on the formula for the sum of the terms of a geometric series, which can be 
found in Theorem | in Section 2.4. 

The iterative approach has produced the solution to the recurrence relation H, = 2H,—1 + 1 
with the initial condition H; = 1. This formula can be proved using mathematical induction. 
This is left for the reader as Exercise 1. 

A myth created to accompany the puzzle tells of a tower in Hanoi where monks are trans- 
ferring 64 gold disks from one peg to another, according to the rules of the puzzle. The myth 
says that the world will end when they finish the puzzle. How long after the monks started will 
the world end if the monks take one second to move a disk? 

From the explicit formula, the monks require 


2% _ 1 = 18,446,744,073,709,551,615 


moves to transfer the disks. Making one move per second, it will take them more than 500 billion 
years to complete the transfer, so the world should survive a while longer than it already has.4 


Remark: Many people have studied variations of the original Tower of Hanoi puzzle discussed 
in Example 2. Some variations use more pegs, some allow disks to be of the same size, and some 
restrict the types of allowable disk moves. One of the oldest and most interesting variations is the 
Reve’s puzzle,* proposed in 1907 by Henry Dudeney in his book The Canterbury Puzzles. The 
Reve’s puzzle involves pilgrims challenged by the Reve to move a stack of cheeses of varying 
sizes from the first of four stools to another stool without ever placing a cheese on one of smaller 
diameter. The Reve’s puzzle, expressed in terms of pegs and disks, follows the same rules as the 


* Reve, more commonly spelled reeve, is an archaic word for governor. 


EXAMPLE 3 
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Tower of Hanoi puzzle, except that four pegs are used. You may find it surprising that no one has 
been able to establish the minimum number of moves required to solve this puzzle for n disks. 
However, there is a conjecture, now more than 50 years old, that the minimum number of moves 
required equals the number of moves used by an algorithm invented by Frame and Stewart 
in 1939. (See Exercises 38-45 and [St94] for more information.) 


Example 3 illustrates how recurrence relations can be used to count bit strings of a specified 
length that have a certain property. 


Find a recurrence relation and give initial conditions for the number of bit strings of length n 
that do not have two consecutive Os. How many such bit strings are there of length five? 


Solution: Let a, denote the number of bit strings of length n that do not have two consecutive Os. 
To obtain a recurrence relation for {a,}, note that by the sum rule, the number of bit strings of 
length n that do not have two consecutive Os equals the number of such bit strings ending with 
a 0 plus the number of such bit strings ending with a |. We will assume that n > 3, so that the 
bit string has at least three bits. 

The bit strings of length n ending with | that do not have two consecutive Os are precisely the 
bit strings of length n — 1 with no two consecutive Os with a | added at the end. Consequently, 
there are a,— such bit strings. 

Bit strings of length n ending with a 0 that do not have two consecutive Os must have 1 
as their (n — 1)st bit; otherwise they would end with a pair of Os. It follows that the bit strings 
of length n ending with a 0 that have no two consecutive Os are precisely the bit strings of 
length n — 2 with no two consecutive Os with 10 added at the end. Consequently, there are a,—2 
such bit strings. 

We conclude, as illustrated in Figure 4, that 


An = An—1 + Gn—-2 


forn > 3. 

The initial conditions are a; = 2, because both bit strings of length one, 0 and 1 do not have 
consecutive Os, and az = 3, because the valid bit strings of length two are 01, 10, and 11. To 
obtain as, we use the recurrence relation three times to find that 


a3=ang+a; =34+2=5, 
a4 =a3+a2 =54+3=8, 
a5 =agt+a3=84+5=13. < 


Number of bit strings 
of length n with no 
two consecutive Os: 


: Any bit string of length n—1 with 1 
End with a 1: no two consecutive Os Gn 
. : Any bit string of length n —2 with 
End with a0: no two consecutive Os 10 Gn-2 
Total: 4G, = 8) Fao 


FIGURE 4 Counting Bit Strings of Length n with No Two Consecutive 0s. 
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EXAMPLE 5 


Remark: Note that {a,,} satisfies the same recurrence relation as the Fibonacci sequence. Because 
a, = f3 and a2 = f4 it follows that a, = fn+2. 


Example 4 shows how a recurrence relation can be used to model the number of codewords 
that are allowable using certain validity checks. 


Codeword Enumeration A computer system considers a string of decimal digits a valid 
codeword if it contains an even number of 0 digits. For instance, 1230407869 is valid, 
whereas 120987045608 is not valid. Let a, be the number of valid n-digit codewords. Find 
a recurrence relation for dy. 


Solution: Note that ay = 9 because there are 10 one-digit strings, and only one, namely, the 
string 0, is not valid. A recurrence relation can be derived for this sequence by considering how 
a valid n-digit string can be obtained from strings of n — 1 digits. There are two ways to form 
a valid string with n digits from a string with one fewer digit. 

First, a valid string of n digits can be obtained by appending a valid string of n — 1 digits 
with a digit other than 0. This appending can be done in nine ways. Hence, a valid string 
with n digits can be formed in this manner in 9a,_1 ways. 

Second, a valid string of n digits can be obtained by appending a 0 to a string of length 
n — | that is not valid. (This produces a string with an even number of 0 digits because the 
invalid string of length n — | has an odd number of 0 digits.) The number of ways that this can 
be done equals the number of invalid (n — 1)-digit strings. Because there are 10’~! strings of 
length n — 1, and ay, _, are valid, there are 10”-! — a,_1 valid n-digit strings obtained by 
appending an invalid string of length n — 1 witha 0. 

Because all valid strings of length n are produced in one of these two ways, it follows that 
there are 


ay = 9an—1 Sw do"! — 4-1) 
= 8a,_; + 10"—! 


valid strings of length n. < 


Example 5 establishes a recurrence relation that appears in many different contexts. 


Find a recurrence relation for C;,, the number of ways to parenthesize the product of n + 1 num- 
bers, x9 + X1 + X2-+ +++ + Xp, to specify the order of multiplication. For example, C3 = 5 because 
there are five ways to parenthesize xg - x; - x2 - x3 to determine the order of multiplication: 


((X0 - X1) » X2) + x3 (xo - (X1 + X2)) + x3 (xo + X1) + (x2 - x3) 
Xo + (x1 + x2) - x3) Xo > (x1 + (x2 + x3)). 


Solution: To develop a recurrence relation for C,, we note that however we insert parentheses 
in the product x9 - x1 -x2- +++ +X, one “-” operator remains outside all parentheses, namely, 
the operator for the final multiplication to be performed. [For example, in (x9 - (x1 - x2)) - x3, 
it is the final “.”, while in (xo - x1) - (x2 - x3) it is the second “-”.] This final operator appears 
between two of the n + 1 numbers, say, xz and xz41. There are CyCy,—~—1 ways to insert 
parentheses to determine the order of the n + 1 numbers to be multiplied when the final op- 
erator appears between x, and x,41, because there are Cy ways to insert parentheses in the 
product xo - x, - ++ - x, to determine the order in which these k + | numbers are to be multi- 
plied and C,,_x—1 ways to insert parentheses in the product xx+1 - xp42- +++ +X» to determine 
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the order in which these n — k numbers are to be multiplied. Because this final operator can 
appear between any two of the n + | numbers, it follows that 


Ch = CoCn—1 + Ci Cp—-2 + +++ + Cy_2C1 + Cn-1Co 


Note that the initial conditions are Co = | and C; = 1. < 


The recurrence relation in Example 5 can be solved using the method of generating func- 
tions, which will be discussed in Section 8.4. It can be shown that C, = C(2n, n)/(n + 1) (see 
Exercise 41 in Section 8.4) and that C, ~ PEN (see [GrKnPa94]). The sequence {C,,} is the 


‘ated sequence of Catalan numbers, named after Eugéne Charles Catalan. This sequence appears 
as the solution of many different counting problems besides the one considered here (see the 
chapter on Catalan numbers in [MiRo91] or [Ro84a] for details). 


Algorithms and Recurrence Relations 


Recurrence relations play an important role in many aspects of the study of algorithms and their 
complexity. In Section 8.3, we will show how recurrence relations can be used to analyze the 
complexity of divide-and-conquer algorithms, such as the merge sort algorithm introduced in 

Section 5.4. As we will see in Section 8.3, divide-and-conquer algorithms recursively divide a 

problem into a fixed number of non-overlapping subproblems until they become simple enough 

to solve directly. We conclude this section by introducing another algorithmic paradigm known 

as dynamic programming, which can be used to solve many optimization problems efficiently. 

An algorithm follows the dynamic programming paradigm when it recursively breaks down 

2 a problem into simpler overlapping subproblems, and computes the solution using the solutions 
Unks of the subproblems. Generally, recurrence relations are used to find the overall solution from 
the solutions of the subproblems. Dynamic programming has been used to solve important 

problems in such diverse areas as economics, computer vision, speech recognition, artificial 
intelligence, computer graphics, and bioinformatics. In this section we will illustrate the use of 

dynamic programming by constructing an algorithm for solving a scheduling problem. Before 

doing so, we will relate the amusing origin of the name dynamic programming, which was 


EUGENE CHARLES CATALAN (1814-1894)  Eugéne Catalan was born in Bruges, then part of France. 
His father became a successful architect in Paris while Eugene was a boy. Catalan attended a Parisian school 
for design hoping to follow in his father’s footsteps. At 15, he won the job of teaching geometry to his design 
school classmates. After graduating, Catalan attended a school for the fine arts, but because of his mathematical 
aptitude his instructors recommended that he enter the Ecole Polytechnique. He became a student there, but after 
his first year, he was expelled because of his politics. However, he was readmitted, and in 1835, he graduated 
and won a position at the Collége de Chalons sur Marne. 

In 1838, Catalan returned to Paris where he founded a preparatory school with two other mathemati- 
cians, Sturm and Liouville. After teaching there for a short time, he was appointed to a position at the Ecole 
Polytechnique. He received his doctorate from the Ecole Polytechnique in 1841, but his political activity in favor of the French 
Republic hurt his career prospects. In 1846 Catalan held a position at the Collége de Charlemagne; he was appointed to the Lycée 
Saint Louis in 1849. However, when Catalan would not take a required oath of allegiance to the new Emperor Louis-Napoleon 
Bonaparte, he lost his job. For 13 years he held no permanent position. Finally, in 1865 he was appointed to a chair of mathematics 
at the University of Liége, Belgium, a position he held until his 1884 retirement. 

Catalan made many contributions to number theory and to the related subject of continued fractions. He defined what are now 
known as the Catalan numbers when he solved the problem of dissecting a polygon into triangles using non-intersecting diagonals. 
Catalan is also well known for formulating what was known as the Catalan conjecture. This asserted that 8 and 9 are the only 
consecutive powers of integers, a conjecture not solved until 2003. Catalan wrote many textbooks, including several that became 
quite popular and appeared in as many as 12 editions. Perhaps this textbook will have a 12th edition someday! 
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introduced by the mathematician Richard Bellman in the 1950s. Bellman was working at the 
RAND Corporation on projects for the U.S. military, and at that time, the U.S. Secretary of 
Defense was hostile to mathematical research. Bellman decided that to ensure funding, he 
needed a name not containing the word mathematics for his method for solving scheduling and 
planning problems. He decided to use the adjective dynamic because, as he said “it’s impossible 
to use the word dynamic in a pejorative sense” and he thought that dynamic programming was 
“something not even a Congressman could object to.” 


AN EXAMPLE OF DYNAMIC PROGRAMMING The problem we use to illustrate dynamic 
programming is related to the problem studied in Example 7 in Section 3.1. In that problem 
our goal was to schedule as many talks as possible in a single lecture hall. These talks have 
preset start and end times; once a talk starts, it continues until it ends; no two talks can proceed 
at the same time; and a talk can begin at the same time another one ends. We developed a 
greedy algorithm that always produces an optimal schedule, as we proved in Example 12 in 
Section 5.1. Now suppose that our goal is not to schedule the most talks possible, but rather to 
have the largest possible combined attendance of the scheduled talks. 

We formalize this problem by supposing that we have n talks, where talk j begins at 
time tj, ends at time ej. and will be attended by Wj students. We want a schedule that maximizes 
the total number of student attendees. That is, we wish to schedule a subset of talks to maximize 
the sum of w; over all scheduled talks. (Note that when a student attends more than one talk, this 
student is counted according to the number of talks attended.) We denote by T (j) the maximum 
number of total attendees for an optimal schedule from the first 7 talks, so T (1) is the maximal 
number of total attendees for an optimal schedule for all n talks. 

We first sort the talks in order of increasing end time. After doing this, we renumber the 
talks so that ey < e2 <--- < e,. We say that two talks are compatible if they can be part of the 
same schedule, that is, if the times they are scheduled do not overlap (other than the possibility 
one ends and the other starts at the same time). We define p(j) to be largest integer i, i < j, 
for which e; < sj, if such an integer exists, and p(j) = 0 otherwise. That is, talk p(j) is the 
talk ending latest among talks compatible with talk 7 that end before talk 7 ends, if such a talk 
exists, and p(j) = 0 if there are no such talks. 


RICHARD BELLMAN (1920-1984) — Richard Bellman, born in Brooklyn, where his father was a grocer, 
spent many hours in the museums and libraries of New York as a child. After graduating high school, he 
studied mathematics at Brooklyn College and graduated in 1941. He began postgraduate work at Johns Hopkins 
University, but because of the war, left to teach electronics at the University of Wisconsin. He was able to continue 
his mathematics studies at Wisconsin, and in 1943 he received his masters degree there. Later, Bellman entered 
Princeton University, teaching in a special U.S. Army program. In late 1944, he was drafted into the army. He 
was assigned to the Manhattan Project at Los Alamos where he worked in theoretical physics. After the war, he 
returned to Princeton and received his Ph.D. in 1946. 

After briefly teaching at Princeton, he moved to Stanford University, where he attained tenure. At 
Stanford he pursued his fascination with number theory. However, Bellman decided to focus on mathematical questions arising from 
real-world problems. In 1952, he joined the RAND Corporation, working on multistage decision processes, operations research 
problems, and applications to the social sciences and medicine. He worked on many military projects while at RAND. In 1965 he 
left RAND to become professor of mathematics, electrical and biomedical engineering and medicine at the University of Southern 
California. 

In the 1950s Bellman pioneered the use of dynamic programming, a technique invented earlier, in a wide range of settings. He 
is also known for his work on stochastic control processes, in which he introduced what is now called the Bellman equation. He 
coined the term curse of dimensionality to describe problems caused by the exponential increase in volume associated with adding 
extra dimensions to a space. He wrote an amazing number of books and research papers with many coauthors, including many on 
industrial production and economic systems. His work led to the application of computing techniques in a wide variety of areas 
ranging from the design of guidance systems for space vehicles, to network optimization, and even to pest control. 

Tragically, in 1973 Bellman was diagnosed with a brain tumor. Although it was removed successfully, complications left him 
severely disabled. Fortunately, he managed to continue his research and writing during his remaining ten years of life. Bellman 
received many prizes and awards, including the first Norbert Wiener Prize in Applied Mathematics and the IEEE Gold Medal of 
Honor. He was elected to the National Academy of Sciences. He was held in high regard for his achievements, courage, and admirable 
qualities. Bellman was the father of two children. 
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8am. 9AM. I10AM. 11AM. 12noon 1PM. 2PM. 3PM. 


FIGURE 5 A Schedule of Lectures with the Values of p() Shown. 


Consider seven talks with these start times and end times, as illustrated in Figure 5. 


Talk 1: start 8 A.M., end 10 A.M. Talk 5: start 8:30 A.M., end 2 P.M. 
Talk 2: start 9 a.M., end 11 A.M. Talk 6: start 11 a.M., end 2 P.M. 
Talk 3: start 10:30 a.m., end 12 noon Talk 7: start 1 P.M., end 2 P.M. 


Talk 4: start 9:30 a.m., end | P.M. 
Find p(j) for j = 1,2,..., 7. 


Solution: We have p(1) = 0 and p(2) = 0, because no talks end before either of the first two 
talks begin. We have p(3) = | because talk 3 and talk 1 are compatible, but talk 3 and talk 2 
are not compatible; p(4) = 0 because talk 4 is not compatible with any of talks 1, 2, and 3; 
p(5) = 0 because talk 5 is not compatible with any of talks 1, 2,3, and 4; and p(6) = 2 because 
talk 6 and talk 2 are compatible, but talk 6 is not compatible with any of talks 3, 4, and 5. Finally, 
P(7) = 4, because talk 7 and talk 4 are compatible, but talk 7 is not compatible with either of 
talks 5 or 6. < 


To develop a dynamic programming algorithm for this problem, we first develop a key 
recurrence relation. To do this, first note that if 7 <n, there are two possibilities for an optimal 
schedule of the first 7 talks (recall that we are assuming that the 7 talks are ordered by increasing 
end time): (i) talk 7 belongs to the optimal schedule or (ii) it does not. 


Case (i): We know that talks p(j) + 1,..., 7 — 1 do not belong to this schedule, for none of 
these other talks are compatible with talk 7. Furthermore, the other talks in this optimal schedule 
must comprise an optimal schedule for talks 1,2,..., p(j). For if there were a better schedule 
for talks 1,2,..., p(j), by adding talk 7, we will have a schedule better than the overall optimal 
schedule. Consequently, in case (i), we have T(j) = w; + T(p(j)). 


Case (ii): When talk j does not belong to an optimal schedule, it follows that an optimal 
schedule from talks 1, 2,..., 7 is the same as an optimal schedule from talks 1,2,..., 7 — 1. 
Consequently, in case (ii), we have T(j) = T(j — 1). Combining cases (i) and (ii) leads us to 
the recurrence relation 


T(j) = max(wj + T(p(y)), TG — 1). 


Now that we have developed this recurrence relation, we can construct an efficient algorithm, 
Algorithm 1, for computing the maximum total number of attendees. We ensure that the algorithm 
is efficient by storing the value of each T (j) after we compute it. This allows us to compute T (/) 
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Exercises 


1. Use mathematical induction to verify the formula derived 


only once. If we did not do this, the algorithm would have exponential worst-case complexity. 
The process of storing the values as each is computed is known as memoization and is an 
important technique for making recursive algorithms efficient. 


ALGORITHM 1 Dynamic Programming Algorithm for Scheduling Talks. 


procedure Maximum Attendees (81, 52,...,5n: start times of talks; 
€1,€2,...,@,: end times of talks; w}, w2,..., W,: number of attendees to talks) 
sort talks by end time and relabel so that ey < e2 <--- < e, 
for j := 1ton 
if no job i withi < j is compatible with job 7 
Pj) =9 
else p(j) := max{i |i < j and jobi is compatible with job j} 
T (0) :=0 
for j := 1 ton 


T(j) = max(wj + T(p(j)), TG — 1) 
return 7 (n) {7 (7) is the maximum number of attendees } 


In Algorithm 1 we determine the maximum number of attendees that can be achieved 
by a schedule of talks, but we do not find a schedule that achieves this maximum. To find 
talks we need to schedule, we use the fact that talk 7 belongs to an optimal solution for the 
first j talks if and only ifw; + T(p(j)) = T(j — 1). We leave it as Exercise 53 to construct an 
algorithm based on this observation that determines which talks should be scheduled to achieve 
the maximum total number of attendees. 

Algorithm | is a good example of dynamic programming as the maximum total atten- 
dance is found using the optimal solutions of the overlapping subproblems, each of which de- 
termines the maximum total attendance of the first 7 talks for some j with | < j<n-1. 
See Exercises 56 and 57 and Supplementary Exercises 14 and 17 for other examples of 
dynamic programming. 


5 pesos, 10 pesos, 20 pesos, 50 pesos, and 100 pesos. Find 


in Example 2 for the number of moves required to com- 
plete the Tower of Hanoi puzzle. 
2. a) Find a recurrence relation for the number of permu- 
tations of a set with n elements. 
b) Use this recurrence relation to find the number of per- 
mutations of a set with n elements using iteration. 


3. A vending machine dispensing books of stamps accepts 
only one-dollar coins, $1 bills, and $5 bills. 

a) Find a recurrence relation for the number of ways 
to deposit n dollars in the vending machine, where 
the order in which the coins and bills are deposited 
matters. 

b) What are the initial conditions? 

c) How many ways are there to deposit $10 for a book 
of stamps? 

4. A country uses as currency coins with values of | peso, 

2 pesos, 5 pesos, and 10 pesos and bills with values of 


*6. 


7. 


a recurrence relation for the number of ways to pay a bill 
of n pesos if the order in which the coins and bills are 
paid matters. 


. How many ways are there to pay a bill of 17 pesos using 


the currency described in Exercise 4, where the order in 
which coins and bills are paid matters? 


a) Find a recurrence relation for the number of strictly 
increasing sequences of positive integers that have 1 
as their first term and n as their last term, where n is 
a positive integer. That is, sequences a), a2,..., dk, 
where a} =1, ag =n, and aj <aj+, for j= 
1,2,...,k—1. 

b) What are the initial conditions? 

c) How many sequences of the type described in (a) are 
there when n is an integer with n > 2? 


a) Find arecurrence relation for the number of bit strings 
of length n that contain a pair of consecutive Os. 


*10. 


11. 


12. 


b) What are the initial conditions? 
c) How many bit strings of length seven contain two 
consecutive 0s? 


a) Find arecurrence relation for the number of bit strings 
of length n that contain three consecutive Os. 

b) What are the initial conditions? 

c) How many bit strings of length seven contain three 
consecutive Os? 


a) Find arecurrence relation for the number of bit strings 
of length n that do not contain three consecutive Os. 

b) What are the initial conditions? 

c) How many bit strings of length seven do not contain 
three consecutive 0s? 


a) Find arecurrence relation for the number of bit strings 
of length n that contain the string 01. 

b) What are the initial conditions? 

c) How many bit strings of length seven contain the 
string 01? 

a) Find a recurrence relation for the number of ways to 
climb n stairs if the person climbing the stairs can take 
one stair or two stairs at a time. 

b) What are the initial conditions? 

c) In how many ways can this person climb a flight of 
eight stairs? 


a) Find a recurrence relation for the number of ways to 
climb n stairs if the person climbing the stairs can take 
one, two, or three stairs at a time. 

b) What are the initial conditions? 

c) In many ways can this person climb a flight of eight 
stairs? 


A string that contains only Os, 1s, and 2s is called a ternary 
string. 


13. 


14. 


*15, 


*16. 


a) Find a recurrence relation for the number of ternary 
strings of length n that do not contain two consecutive 
Os. 

b) What are the initial conditions? 

c) How many ternary strings of length six do not contain 
two consecutive Os? 


a) Find a recurrence relation for the number of 
ternary strings of length n that contain two 
consecutive Os. 

b) What are the initial conditions? 

c) How many ternary strings of length six contain two 
consecutive 0s? 


a) Find a recurrence relation for the number of ternary 
strings of length n that do not contain two consecutive 
Os or two consecutive Is. 

b) What are the initial conditions? 

c) How many ternary strings of length six do not contain 
two consecutive Os or two consecutive 1s? 


a) Find a recurrence relation for the number of ternary 
strings of length n that contain either two consecutive 
Os or two consecutive Is. 

b) What are the initial conditions? 

c) How many ternary strings of length six contain two 
consecutive Os or two consecutive 1s? 


*17, 


** 18, 


19. 


20. 


21. 


*22, 


* 23. 


24. 


25. 


8.1 Applications of Recurrence Relations 511 


a) Find a recurrence relation for the number of ternary 
strings of length n that do not contain consecutive 
symbols that are the same. 


b) What are the initial conditions? 


c) How many ternary strings of length six do not contain 
consecutive symbols that are the same? 


a) Find a recurrence relation for the number of ternary 
strings of length n that contain two consecutive sym- 
bols that are the same. 


b) What are the initial conditions? 


c) How many ternary strings of length six contain con- 
secutive symbols that are the same? 


Messages are transmitted over a communications channel 
using two signals. The transmittal of one signal requires 
1 microsecond, and the transmittal of the other signal re- 
quires 2 microseconds. 


a) Find a recurrence relation for the number of differ- 
ent messages consisting of sequences of these two 
signals, where each signal in the message is imme- 
diately followed by the next signal, that can be sent 
in n microseconds. 


b) What are the initial conditions? 


c) How many different messages can be sent in 10 mi- 
croseconds using these two signals? 


A bus driver pays all tolls, using only nickels and dimes, 
by throwing one coin at a time into the mechanical toll 
collector. 


a) Find arecurrence relation for the number of different 
ways the bus driver can pay a toll of n cents (where 
the order in which the coins are used matters). 


b) In how many different ways can the driver pay a toll 
of 45 cents? 


a) Find the recurrence relation satisfied by R,,, where R, 
is the number of regions that a plane is divided into 
by n lines, if no two of the lines are parallel and no 
three of the lines go through the same point. 


b) Find R,, using iteration. 


a) Find the recurrence relation satisfied by R,,, where R, 
is the number of regions into which the surface of a 
sphere is divided by n great circles (which are the in- 
tersections of the sphere and planes passing through 
the center of the sphere), if no three of the great circles 
go through the same point. 

b) Find R,, using iteration. 

a) Find the recurrence relation satisfied by S,, where S,, 
is the number of regions into which three-dimensional 
space is divided by n planes if every three of the planes 
meet in one point, but no four of the planes go through 
the same point. 

b) Find S,, using iteration. 

Find a recurrence relation for the number of bit sequences 

of length n with an even number of Os. 


How many bit sequences of length seven contain an even 
number of Os? 
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26. a) Find a recurrence relation for the number of ways to 
completely cover a 2 x n checkerboard with 1 x 2 
dominoes. [Hint: Consider separately the coverings 
where the position in the top right corner of the 
checkerboard is covered by a domino positioned hor- 
izontally and where it is covered by a domino posi- 
tioned vertically.] 

b) What are the initial conditions for the recurrence re- 

lation in part (a)? 

How many ways are there to completely cover a 

2 x 17 checkerboard with 1 x 2 dominoes? 


c) 


27. a) Find a recurrence relation for the number of ways to 

lay out a walkway with slate tiles if the tiles are red, 

green, or gray, so that no two red tiles are adjacent and 

tiles of the same color are considered indistinguish- 

able. 

b) What are the initial conditions for the recurrence re- 
lation in part (a)? 

c) How many ways are there to lay out a path of seven 
tiles as described in part (a)? 


28. Show that the Fibonacci numbers satisfy the recurrence 
relation fp =5fn-a+3fn—s for n =5,6,7,..., to- 
gether with the initial conditions fo = 0, fi = 1, fo = 1, 
f3 = 2, and f4 = 3. Use this recurrence relation to show 


that fs, is divisible by 5, forn = 1,2,3,.... 


Let S(m,n) denote the number of onto functions from 
a set with m elements to a set with n elements. Show 
that S(m, n) satisfies the recurrence relation 


* 29, 


n—-l 
S(m,n) =n" — Y°C(n, k)S(m, k) 
k=1 


whenever m > n and n > 1, with the initial condition 
S(m, 1) = 1. 

a) Write out all the ways the product xo - x1 - x2 -x3 + x4 
can be parenthesized to determine the order of multi- 
plication. 

Use the recurrence relation developed in Example 5 
to calculate C4, the number of ways to parenthesize 
the product of five numbers so as to determine the or- 
der of multiplication. Verify that you listed the correct 
number of ways in part (a). 

Check your result in part (b) by finding C4, using the 
closed formula for C, mentioned in the solution of 
Example 5. 


30. 


b) 


c) 


31. a) Use the recurrence relation developed in Example 5 to 
determine Cs, the number of ways to parenthesize the 
product of six numbers so as to determine the order 
of multiplication. 

Check your result with the closed formula for C5 men- 


tioned in the solution of Example 5. 


b) 


* 32. In the Tower of Hanoi puzzle, suppose our goal is to trans- 
fer all n disks from peg 1 to peg 3, but we cannot move a 
disk directly between pegs | and 3. Each move of a disk 
must be a move involving peg 2. As usual, we cannot 


place a disk on top of a smaller disk. 


CG 


a) Find arecurrence relation for the number of moves re- 
quired to solve the puzzle for n disks with this added 
restriction. 

Solve this recurrence relation to find a formula for the 
number of moves required to solve the puzzle for n 
disks. 

How many different arrangements are there of the n 
disks on three pegs so that no disk is on top of a smaller 
disk? 

Show that every allowable arrangement of the n disks 
occurs in the solution of this variation of the puzzle. 


b) 


c) 


d) 


Exercises 33-37 deal with a variation of the Josephus 
problem described by Graham, Knuth, and Patashnik in 
[GrKnPa94]. This problem is based on an account by the his- 
torian Flavius Josephus, who was part of a band of 41 Jewish 
rebels trapped in a cave by the Romans during the Jewish- 
Roman war of the first century. The rebels preferred suicide 
to capture; they decided to form a circle and to repeatedly 
count off around the circle, killing every third rebel left alive. 
However, Josephus and another rebel did not want to be killed 
this way; they determined the positions where they should 
stand to be the last two rebels remaining alive. The variation 
we consider begins with n people, numbered 1 to n, stand- 
ing around a circle. In each stage, every second person still 
left alive is eliminated until only one survives. We denote the 
number of the survivor by J(n). 

33. Determine the value of J(n) for each integer n with 
l<n< 16. 

Use the values you found in Exercise 33 to conjecture a 
formula for J(n). [Hint: Write n = 2 +k, where m is 
a nonnegative integer and k is a nonnegative integer less 
than 2”".] 

Show that J (n) satisfies the recurrence relation J(2n) = 
2J(n) — 1 and J(2n + 1) = 2J/(n) +1, for n > 1, and 
Jd) =1. 

Use mathematical induction to prove the formula you 
conjectured in Exercise 34, making use of the recurrence 
relation from Exercise 35. 

Determine J(100), J(1000), and J(10,000) from your 
formula for J (7). 


Exercises 38-45 involve the Reve’s puzzle, the variation of 
the Tower of Hanoi puzzle with four pegs and n disks. Before 
presenting these exercises, we describe the Frame—Stewart al- 
gorithm for moving the disks from peg 1 to peg 4 so that no 
disk is ever on top of a smaller one. This algorithm, given 
the number of disks n as input, depends on a choice of an 
integer k with 1 < k <n. When there is only one disk, move 
it from peg 1 to peg 4 and stop. Forn > 1, the algorithm pro- 
ceeds recursively, using these three steps. Recursively move 
the stack of the n — k smallest disks from peg 1 to peg 2, 
using all four pegs. Next move the stack of the k largest 
disks from peg | to peg 4, using the three-peg algorithm from 
the Tower of Hanoi puzzle without using the peg holding 
the n—k smallest disks. Finally, recursively move the 
smallest n — k disks to peg 4, using all four pegs. Frame 
and Stewart showed that to produce the fewest moves using 
their algorithm, k should be chosen to be the smallest integer 


34. 


35. 


36. 


37. 


such that n does not exceed t, = k(k + 1)/2, the kth triangu- 
lar number, that is, t,_; <n < t,. The unsettled conjecture, 
known as Frame’s conjecture, is that this algorithm uses the 
fewest number of moves required to solve the puzzle, no mat- 
ter how the disks are moved. 


38. 


39. 


40. 


*41, 


* 42, 


* 43. 


*44, 


*45, 


Show that the Reve’s puzzle with three disks can be solved 
using five, and no fewer, moves. 


Show that the Reve’s puzzle with four disks can be solved 
using nine, and no fewer, moves. 


Describe the moves made by the Frame—Stewart al- 
gorithm, with k chosen so that the fewest moves are 
required, for 

a) 5disks. b) 6disks. cc) 7disks. 4d) 8 disks. 
Show that if R(m) is the number of moves used by 
the Frame—Stewart algorithm to solve the Reve’s puzzle 
with n disks, where k is chosen to be the smallest integer 
with n < k(k + 1)/2, then R(n) satisfies the recurrence 
relation R(n) =2R(n —k) +2* —1, with R(0) =0 
and R(1) = 1. 


Show that if k is as chosen in Exercise 41, then 
R(n) — Rm—1) =2*-!, 

Show that if k is as chosen in Exercise 41, then 
R= HG emo. 


Use Exercise 43 to give an upper bound on the num- 
ber of moves required to solve the Reve’s puzzle for all 
integers n with 1 <n < 25. 


Show that R(n) is O(/n2¥2"). 


Let {a,} be a sequence of real numbers. The backward dif- 
ferences of this sequence are defined recursively as shown 
next. The first difference Va, is 


Vay = An — An_-1- 


The (k + 1)st difference vitla, is obtained from Vian by 


46. 


47. 
48. 
49. 
*50. 


51. 


52. 


ve la, = Via, - Vi an-1. 


Find Va, for the sequence {a,,}, where 

a) a, = 4. b) dy) = 2n. 

c) ay =n’. d) dy = 2". 

Find V7a, for the sequences in Exercise 46. 

Show that a,_1 = ad, — Vay. 

Show that a)_2 = dy — 2Vay + Vp. 

Prove that a,_, can be expressed in terms of ay, Van, 


V2an, sees Ve an. 


Express the recurrence relation a, = a,—; + d,—2 in 
terms of ay, Van, and V7ay. 


Show that any recurrence relation for the sequence {a,} 
can be written in terms of an, Van, V2an, .... The result- 
ing equation involving the sequences and its differences 
is called a difference equation. 


* 53. 


54. 


55. 


56. 


* 57, 
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Construct the algorithm described in the text after Algo- 
rithm 1 for determining which talks should be scheduled 
to maximize the total number of attendees and not just 
the maximum total number of attendees determined by 
Algorithm 1. 


Use Algorithm 1 to determine the maximum number of 
total attendees in the talks in Example 6 if w;, the number 
of attendees of talk i,i = 1,2,...,7, is 


a) 20, 10, 50, 30, 15, 25, 40. 

b) 100, 5, 10, 20, 25, 40, 30. 

c) 2,3, 8,5, 4, 7, 10. 

d) 10, 8, 7, 25, 20, 30, 5. 

For each part of Exercise 54, use your algorithm from 


Exercise 53 to find the optimal schedule for talks so that 
the total number of attendees is maximized. 


In this exercise we will develop a dynamic program- 
ming algorithm for finding the maximum sum of con- 
secutive terms of a sequence of real numbers. That 
is, given a sequence of real numbers aj, a2,..., dn, 
the algorithm computes the maximum sum pee jai 
where 1 <j <k <n. 

a) Show that if all terms of the sequence are nonnegative, 
this problem is solved by taking the sum of all terms. 
Then, give an example where the maximum sum of 
consecutive terms is not the sum of all terms. 

b) Let M(k) be the maximum of the sums of consecutive 
terms of the sequence ending at a;,. That is, M(k) = 
Max] <j<k Yor . a;. Explain why the recurrence rela- 
tion M(k) = max(M(k — 1) + ag, ag) holds for k = 
Die Ie 

c) Use part (b) to develop a dynamic programming algo- 
rithm for solving this problem. 

d) Show each step your algorithm from part (c) uses to 
find the maximum sum of consecutive terms of the 
sequence 2, —3, 4, 1, —2, 3. 

e) Show that the worst-case complexity in terms of the 
number of additions and comparisons of your algo- 
rithm from part (c) is linear. 


Dynamic programming can be used to develop 
an algorithm for solving the matrix-chain multi- 
plication problem introduced in Section 3.3. This 
is the problem of determining how the product 
AjA2---A, can be computed using the fewest 
integer multiplications, where Aj,,A2,...,A, are 
my, X mM2,m2 X M3,..., Mn X Myn41 Matrices, respec- 
tively, and each matrix has integer entries. Recall that 
by the associative law, the product does not depend on 
the order in which the matrices are multiplied. 


a) Show that the brute-force method of determining the 
minimum number of integer multiplications needed to 
solve a matrix-chain multiplication problem has expo- 
nential worst-case complexity. [Hint: Do this by first 
showing that the order of multiplication of matrices 
is specified by parenthesizing the product. Then, use 
Example 5 and the result of part (c) of Exercise 41 in 
Section 8.4.] 
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b) Denote by Aj; the product AjAj+1..., Aj, c) Explain why part (b) leads to the recurrence rela- 
and M(i, j) the minimum number of integer mul- tion MG, j) = minjcg<j(M(i,k) + MK + 1, jf) + 
tiplications required to find A;;. Show that if the mimgimjpyifl<isj<jsn. 
least number of integer multiplications are used to d) Use the recurrence relation in part (c) to construct 
compute A;;, where i < j, by splitting the product an efficient algorithm for determining the order 
into the product of A; through A, and the product the n matrices should be multiplied to use the min- 
of Ax41 through Aj, then the first k terms must imum number of integer multiplications. Store the 
be parenthesized so that Aj, is computed in the partial results M(i, j) as you find them so that your 
optimal way using M(i,k) integer multiplications algorithm will not have exponential complexity. 
and Ax+;,; must be parenthesized so that Aj+1,; e) Show that your algorithm from part (d) has O(n?) 


is computed in the optimal way using M(k + 1, j) 
integer multiplications. 


worst-case complexity in terms of multiplications of 
integers. 


Solving Linear Recurrence Relations 


Links 


DEFINITION 1 


EXAMPLE 1 


EXAMPLE 2 


Introduction 


A wide variety of recurrence relations occur in models. Some of these recurrence relations can 
be solved using iteration or some other ad hoc technique. However, one important class of 
recurrence relations can be explicitly solved in a systematic way. These are recurrence relations 
that express the terms of a sequence as linear combinations of previous terms. 


A linear homogeneous recurrence relation of degree k with constant coefficients is a recut- 
rence relation of the form 


An = C{An—1 + C2Qn—-2 + +++ + Ckan—k, 


where C1, C2, ..., Cx are real numbers, and cy, # 0. 


The recurrence relation in the definition is linear because the right-hand side is a sum of 
previous terms of the sequence each multiplied by a function of n. The recurrence relation is 
homogeneous because no terms occur that are not multiples of the a;s. The coefficients of the 
terms of the sequence are all constants, rather than functions that depend on n. The degree 
is k because ay, is expressed in terms of the previous & terms of the sequence. 

A consequence of the second principle of mathematical induction is that a sequence satis- 
fying the recurrence relation in the definition is uniquely determined by this recurrence relation 
and the k initial conditions 


a9 = Co, a) = Cj, ..-, Qk-1 = Ce-1- 


The recurrence relation P, = (1.11) P,—1 is a linear homogeneous recurrence relation of degree 
one. The recurrence relation f;, = f,—1 + fn—2 is a linear homogeneous recurrence relation of 
degree two. The recurrence relation a, = dy—s is a linear homogeneous recurrence relation of 
degree five. < 


Example 2 presents some examples of recurrence relations that are not linear homogeneous 
recurrence relations with constant coefficients. 


The recurrence relation a, = d,—1 + or is not linear. The recurrence relation H, = 
2H,—1 + 1 is not homogeneous. The recurrence relation B, = nB,—, does not have constant 
coefficients. < 


THEOREM 1 
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Linear homogeneous recurrence relations are studied for two reasons. First, they often occur 
in modeling of problems. Second, they can be systematically solved. 


Solving Linear Homogeneous Recurrence Relations 
with Constant Coefficients 


The basic approach for solving linear homogeneous recurrence relations is to look for solutions 
of the form a, = r”, where r is a constant. Note that a, = r” is a solution of the recurrence 
relation dy = C{dyn—1 + C2An—2 +--+ + CKan— if and only if 


r® = cpr! 4 egr™ 2 4. + er. 


When both sides of this equation are divided by r”~* 
the left, we obtain the equation 


and the right-hand side is subtracted from 


k-2 


r® — cyr®! — cor =“ ear = cE = 0), 


Consequently, the sequence {a,} with a, =r” is a solution if and only if r is a solution of this 
last equation. We call this the characteristic equation of the recurrence relation. The solutions 
of this equation are called the characteristic roots of the recurrence relation. As we will see, 
these characteristic roots can be used to give an explicit formula for all the solutions of the 
recurrence relation. 

We will first develop results that deal with linear homogeneous recurrence relations with 
constant coefficients of degree two. Then corresponding general results when the degree may be 
greater than two will be stated. Because the proofs needed to establish the results in the general 
case are more complicated, they will not be given here. 

We now turn our attention to linear homogeneous recurrence relations of degree two. First, 
consider the case when there are two distinct characteristic roots. 


Let c; and cz be real numbers. Suppose that r2 — cir — c7 = 0 has two distinct roots r} 
and r2. Then the sequence {a,} is a solution of the recurrence relation dy = cjan—1 + c2an~2 
if and only if a, = ar} + a2rz forn =0,1,2,..., where a and a are constants. 


Proof: We must do two things to prove the theorem. First, it must be shown that if ry and r2 
are the roots of the characteristic equation, and a; and a are constants, then the sequence {a,,} 
with a, = ayr} + @2r5 is a solution of the recurrence relation. Second, it must be shown that 
if the sequence {d,} is a solution, then a, = ar} + ary for some constants a and a. 

Now we will show that if a, = ajrj +a2r5, then the sequence {a,} is a solution 
of the recurrence relation. Because rj and rz are roots of r2 —cjr —c2 = 0, it follows 
that rf = cr, +2, re = Ciro + Cp. 

From these equations, we see that 


-1 -1 —2 —2 
€14n—1 + €2Q@n-2 = (air + aery") + co(miry * + art“) 
—2 —2 
=ayr} (cir) + c2) + aor“ (c1r2 + €2) 
n—2_.2 n—2_.2 


Hay TY + O2ry “15 
= ar} + aor; 


= an. 


This shows that the sequence {a,} witha, = a rt + ars is a solution of the recurrence relation. 
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To show that every solution {a,} of the recurrence relation ay, = cja,—, + c2dn~2 
has dy, = ary + ors forn = 0,1, 2,..., for some constants a; and a2, suppose that {a,} is a 
solution of the recurrence relation, and the initial conditions ag = Cop and a; = C, hold. It will 
be shown that there are constants a and a2 such that the sequence {ay} with ay = ayrj + a2r7 
satisfies these same initial conditions. This requires that 


ag = Co =O + ap, 
ay = Cy = ar, + a2r2. 


We can solve these two equations for a; and a2. From the first equation it follows that 
a2 = Co — a. Inserting this expression into the second equation gives 


Cy =ayry + (Co — a1 )r2. 
Hence, 
Cy =ay(r) — r2) + Core. 


This shows that 


C; — Core 
nn 


a) 


and 


Cy —Cor2 Corrs —Ci 
nNo-nm  nmo-n ’ 


a2 = Co—ay = Co- 


where these expressions for a; and a2 depend on the fact that r} 4 r2. (When 7 = ro, this 
theorem is not true.) Hence, with these values for a; and w, the sequence {a,} with ar + a2r7 
satisfies the two initial conditions. 

We know that {a,} and {a rj +a2r5} are both solutions of the recurrence relation 
An = C1An—| + C2d,~2 and both satisfy the initial conditions when n = 0 and n = 1. Because 
there is a unique solution of a linear homogeneous recurrence relation of degree two with two 
initial conditions, it follows that the two solutions are the same, that is, dy = ar} + a2rz for 
all nonnegative integers n. We have completed the proof by showing that a solution of the lin- 
ear homogeneous recurrence relation with constant coefficients of degree two must be of the 
form ay, = ar} + a2rz, where a and @ are constants. <J 


The characteristic roots of a linear homogeneous recurrence relation with constant coeffi- 
cients may be complex numbers. Theorem | (and also subsequent theorems in this section) still 
applies in this case. Recurrence relations with complex characteristic roots will not be discussed 
in the text. Readers familiar with complex numbers may wish to solve Exercises 38 and 39. 

Examples 3 and 4 show how to use Theorem | to solve recurrence relations. 


What is the solution of the recurrence relation 
Ay = An—1 + 2an—2 
with ag = 2 anda, = 7? 
Solution: Theorem 1 can be used to solve this problem. The characteristic equation of the 
recurrence relation is r2 — r — 2 = 0. Its roots are r = 2 andr = —1. Hence, the sequence {a,} 


is a solution to the recurrence relation if and only if 


An = 012" + a2(—1)", 


EXAMPLE 4 
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for some constants a; and a. From the initial conditions, it follows that 


a9 =2=0,4+ a, 
aq, =7=a,-2+a2- (1). 


Solving these two equations shows that wa; = 3 and a2 = —1. Hence, the solution to the recur- 
rence relation and initial conditions is the sequence {a,} with 


an = 3-2" —(-1)". < 


Find an explicit formula for the Fibonacci numbers. 


Solution: Recall that the sequence of Fibonacci numbers satisfies the recurrence relation 
Sn = fn—1 + fn—2 and also satisfies the initial conditions fo = 0 and f; = 1. The roots of the 
characteristic equation r? — r — 1 = Oarer) = (1+ J5)/2 and r2 = (1 — /5)/2. Therefore, 
from Theorem | it follows that the Fibonacci numbers are given by 


fn = 1 (4) +m (=) ; 


for some constants a, and a2. The initial conditions fg = 0 and f; = 1 can be used to find these 
constants. We have 


fo =a + a2 =0, 


fi =o (+) +m (45%) =1 


The solution to these simultaneous equations for a; and a is 


aj = 1/V5, a = —1/V5. 


Consequently, the Fibonacci numbers are given by 


ree ee "1 (1-v5\" 
" V5\ 2 Vs\ 2 } ’ 
Theorem | does not apply when there is one characteristic root of multiplicity two. If 


this happens, then a, = nrg is another solution of the recurrence relation when rp is a root of 
multiplicity two of the characteristic equation. Theorem 2 shows how to handle this case. 


Let c; and c2 be real numbers with c2 4 0. Suppose that r= cir — c2 = 0 has only one 
root ro. A sequence {a,} is a solution of the recurrence relation dy = c1dy,—1 + C2dy~—2 if and 
only if dy = arg + agro, forn = 0, 1,2,..., where a; and q are constants. 


The proof of Theorem 2 is left as Exercise 10. Example 5 illustrates the use of this theorem. 
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ay = 6ay—1 —_ 9ay—2 
with initial conditions ag = | and a, = 6? 


Solution: The only root of r? — 6r +9 = 0 is r = 3. Hence, the solution to this recurrence 
relation is 


An = 013" + ann3” 
for some constants a; and a. Using the initial conditions, it follows that 


a9 =l=a, 
a, =6=a,-3+4+a)-3. 


Solving these two equations shows that a; = | and a2 = 1. Consequently, the solution to this 
recurrence relation and the initial conditions is 


An = 3" +73". 4 


We will now state the general result about the solution of linear homogeneous recurrence 
relations with constant coefficients, where the degree may be greater than two, under the as- 
sumption that the characteristic equation has distinct roots. The proof of this result will be left 
as Exercise 16. 


Let c,, C2, ..., cg be real numbers. Suppose that the characteristic equation 
foci ep — 0 
has k distinct roots r}, 72, ..., rr. Then a sequence {a,,} is a solution of the recurrence relation 


An = C{An—| + C2Qn—2 + +++ + CKAn—k 
if and only if 
An = Ory + gry +--+ + Org 


forn =0,1,2,..., where a, @2,..., % are constants. 


We illustrate the use of the theorem with Example 6. 


EXAMPLE 6 | Find the solution to the recurrence relation 


An = 64y—1 — Llay—2 + 6an—3 


with the initial conditions ag = 2, a; = 5, and az = 15. 
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Solution: The characteristic polynomial of this recurrence relation is 
r? — 6r2 + 11r —6. 


The characteristic roots are r=1l1, r=2, and r=3, because r?—6r2+ llr -—6= 
(r — 1)(r — 2)(r — 3). Hence, the solutions to this recurrence relation are of the form 


An =a, - 1" +a0-2” +03 -3". 
To find the constants a, a2, and a3, use the initial conditions. This gives 


a =2=a;+a2+ 43, 
aq, =5=a;,+a2-2+a3-3, 
az = 15 =a, +a2-44+ 03-9. 


When these three simultaneous equations are solved for a1, a2, and a3, we find that a; = 1, 
a2 = —1, and a3 = 2. Hence, the unique solution to this recurrence relation and the given initial 
conditions is the sequence {a,,} with 


a =1—2"4+2-3". < 


We now state the most general result about linear homogeneous recurrence relations with 
constant coefficients, allowing the characteristic equation to have multiple roots. The key point 
is that for each root r of the characteristic equation, the general solution has a summand of the 
form P(n)r", where P(n) is a polynomial of degree m — 1, with m the multiplicity of this root. 
We leave the proof of this result as Exercise 51. 


Let cj, c2, ..., ck be real numbers. Suppose that the characteristic equation 
Por == oO 

has ¢ distinct roots rj,72,...,7; With multiplicities m,,mz2,...,m,, respectively, so 

that mj; > 1 fori =1,2,...,tf and mj +m2+---+m,; =k. Then a sequence {a,} is a 


solution of the recurrence relation 
Gn = C{An—| + C2Gn—2 + +++ + CKan—k 
if and only if 


an = (1,0 IP ayn Spoor. Cima re 
+ (02,0 + 02,10 + +++ + 2,m.—10— |r? 


Bea CH) ters iA mec a te a ner ie a)ye 
forn = 0,1,2,..., where a;,; are constants for 1 <i <tand0O < j <m;—1. 


Example 7 illustrates how Theorem 4 is used to find the general form of a solution of a 
linear homogeneous recurrence relation when the characteristic equation has several repeated 
roots. 
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Suppose that the roots of the characteristic equation of a linear homogeneous recurrence relation 
are 2, 2, 2, 5,5, and 9 (that is, there are three roots, the root 2 with multiplicity three, the root 
5 with multiplicity two, and the root 9 with multiplicity one). What is the form of the general 
solution? 

Solution: By Theorem 4, the general form of the solution is 


(1,9 + ay,1n + 04,2n7)2” + (a2,9 + @2,12)5” + 013,09". < 


We now illustrate the use of Theorem 4 to solve a linear homogeneous recurrence relation 
with constant coefficients when the characteristic equation has a root of multiplicity three. 


Find the solution to the recurrence relation 


an = —3an | = 3a 2 — an-3 
with initial conditions ag = 1, ay = —2, and az = —1. 
Solution: The characteristic equation of this recurrence relation is 

3 2 = 

r-+3r°+3r+1=0. 
Because r? + 3r? + 3r + 1 = (r + 1)’, there is a single root r = —1 of multiplicity three of 
the characteristic equation. By Theorem 4 the solutions of this recurrence relation are of the 
form 

Gn = o1,9(—1)" + 0 1n(—1)” + a, 2? (-1)". 


To find the constants @1,9, @1,1, and a2, use the initial conditions. This gives 


ag = 1= Q10; 


a) = —2 = 1,90 — 1,1 — 1,2, 
az = —1=a1,9 + 20,1 + 40,2. 
The simultaneous solution of these three equations is a9 = 1, a1.) = 3, and a}. = —2. 


Hence, the unique solution to this recurrence relation and the given initial conditions is the 
sequence {a,} with 


_ 2 n 
ayn = (1 + 3n — 2n*)(- 1)". < 


Linear Nonhomogeneous Recurrence Relations 
with Constant Coefficients 


We have seen how to solve linear homogeneous recurrence relations with constant coefficients. 
Is there a relatively simple technique for solving a linear, but not homogeneous, recurrence 
relation with constant coefficients, such as dy = 3a,—, + 2n? We will see that the answer is yes 
for certain families of such recurrence relations. 

The recurrence relation a, = 3da,—,| + 2n is an example of a linear nonhomogeneous 
recurrence relation with constant coefficients, that is, a recurrence relation of the form 


An = C1An—1 + C2Qn—-2 + +++ + CKAn—k + F(n), 
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where C1, C2,..., Cx are real numbers and F(n) is a function not identically zero depending 
only on n. The recurrence relation 


An = C{An—1 + C2Gn—2 + +++ + CKAn—k 


is called the associated homogeneous recurrence relation. It plays an important role in the 
solution of the nonhomogeneous recurrence relation. 


Each of the recurrence relations a) = dy—1 +2", dn =Qn—1 tQn_2 +n? +n4+1, an = 
3a,_-1 +3”, and ay = dy—| + Gy-2 + a,_-3 +n! is a linear nonhomogeneous recurrence re- 
lation with constant coefficients. The associated linear homogeneous recurrence relations are 
An = An—1, An = An—1 + An—2, An = 3dn—1, aNd Ay = An—1 + An—2 + Gn—3, respectively. <4 


The key fact about linear nonhomogeneous recurrence relations with constant coefficients 


is that every solution is the sum of a particular solution and a solution of the associated linear 
homogeneous recurrence relation, as Theorem 5 shows. 


If {al? 4 is a particular solution of the nonhomogeneous linear recurrence relation with 
constant coefficients 


An = C1An—1 + C2Gn—2 + +++ + CKan—k + F(n), 


then every solution is of the form fal? 4 al}, where {al} is a solution of the associated 


homogeneous recurrence relation 
An = C1An—1 + C24n—2 + +++ + CKan—k. 
Proof: Because {af? , } is a particular solution of the nonhomogeneous recurrence relation, we 
know that 


an? = cya”, + coayP'y +--+ + ceay?, + F(n). 


Now suppose that {b,,} is a second solution of the nonhomogeneous recurrence relation, so that 
by = Ci Dn—1 + Cobp—2 +++ + CKbDn—K + Fn). 
Subtracting the first of these two equations from the second shows that 


bn — a = c1(bp-1 — a?) + €2(bn—-2 — as) +e + CK (On—k = a). 


n n 


It follows that {b, —a?} is a solution of the associated homogeneous linear recurrence, 


say, fa}. Consequently, b, = al? ) + a for all n. | 


By Theorem 5, we see that the key to solving nonhomogeneous recurrence relations with 
constant coefficients is finding a particular solution. Then every solution is a sum of this solution 
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and a solution of the associated homogeneous recurrence relation. Although there is no general 
method for finding such a solution that works for every function F'(n), there are techniques that 
work for certain types of functions F'(7), such as polynomials and powers of constants. This is 
illustrated in Examples 10 and 11. 


Find all solutions of the recurrence relation a, = 3a,—, + 2n. What is the solution with a, = 3? 


Solution: To solve this linear nonhomogeneous recurrence relation with constant coefficients, we 
need to solve its associated linear homogeneous equation and to find a particular solution for the 
given nonhomogeneous equation. The associated linear homogeneous equation is dy = 3an—1. 
Its solutions are a”) = a3", where a is a constant. 

We now find a particular solution. Because F(n) = 2n is a polynomial in n of degree 
one, a reasonable trial solution is a linear function in n, say, py = cn + d, where c and d are 
constants. To determine whether there are any solutions of this form, suppose that p, = cn + dis 
such a solution. Then the equation a, = 3a,_; + 2n becomes cn +d =3(c(n — 1) +d) + 2n. 
Simplifying and combining like terms gives (2 + 2c)n + (2d — 3c) = 0. It follows that cn + d 
is a solution if and only if 2 + 2c = 0 and 2d — 3c = 0. This shows that cn + d is a solution if 
and only if c = —1 and d = —3/2. Consequently, al?) = —n — 3/2 1s a particular solution. 

By Theorem 5 all solutions are of the form 


3 
dn =a +a =-n- ee 


where @ is a constant. 

To find the solution with a; = 3, let n = | in the formula we obtained for the general 
solution. We find that 3 = —1 — 3/2 + 3a, which implies that aw = 11/6. The solution we seek 
is dy = —n — 3/24 (11/6)3". < 


Find all solutions of the recurrence relation 


An = 5an—1 — 64n-2 + 7". 


Solution: This is a linear nonhomogeneous recurrence relation. The solutions of its associated 
homogeneous recurrence relation 


Gy = San—1 — OAn—2 
are a) =a, -3"+a2-2”, where a; and a are constants. Because F(n) = 7", a reason- 
able trial solution is al? =C. 7", where C is a constant. Substituting the terms of this se- 
quence into the recurrence relation implies that C - 7” = 5C -7”~! — 6C - 7"-? +7". Factoring 
out 7”~2, this equation becomes 49C = 35C — 6C + 49, which implies that 20C = 49, or that 


C = 49/20. Hence, al? = (49/20)7” is a particular solution. By Theorem 5, all solutions are 
of the form 


ayn = a + 3" + a2 - 2” + (49/20)7". < 


In Examples 10 and 11, we made an educated guess that there are solutions of a particular 
form. In both cases we were able to find particular solutions. This was not an accident. Whenever 
F (n) is the product of a polynomial inn and the nth power of a constant, we know exactly what 
form a particular solution has, as stated in Theorem 6. We leave the proof of Theorem 6 as 
Exercise 52. 
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Suppose that {a,,} satisfies the linear nonhomogeneous recurrence relation 
An = C1An—1 + C2An—2 + +++ + CKan—K + F(n), 

where c}, C2,..., Cx are real numbers, and 
F(n) = (bn! + by-yn'! +--+ bin + bo)s”, 


where bo, by, ..., b; ands are real numbers. When s is not a root of the characteristic equation 
of the associated linear homogeneous recurrence relation, there is a particular solution of the 
form 


(pen! + prin’ | +---+ pin + po)s”. 


When s is a root of this characteristic equation and its multiplicity is m, there is a particular 
solution of the form 


n™ (pin! + py_in’ | +--+ pin + po)s”. 


Note that in the case when s is a root of multiplicity m of the characteristic equation of 
the associated linear homogeneous recurrence relation, the factor n” ensures that the proposed 
particular solution will not already be a solution of the associated linear homogeneous recurrence 
relation. We next provide Example 12 to illustrate the form of a particular solution provided by 
Theorem 6. 


What form does a particular solution of the linear nonhomogeneous recurrence relation 
Ayn = 64y—1 — 94n—2 + F(n) have when F(n) = 3", F(n) = n3", F(n) = n22", and F(n) = 
(n? + 1)3"? 


Solution: The associated linear homogeneous recurrence relation is a, = 6day,—| — 9an—2. Its 
characteristic equation, r? — 6r + 9 = (r — 3)? = 0, has a single root, 3, of multiplicity two. 
To apply Theorem 6, with F(n) of the form P(n)s”, where P(n) is a polynomial and s is a 
constant, we need to ask whether s is a root of this characteristic equation. 

Because s = 3isaroot with multiplicity m = 2 buts = 2isnotaroot, Theorem 6 tells us that 
a particular solution has the form pon?3” if F(n) = 3”, the form n>(pin + po)3” if F(n) = 
n3", the form (p2n” + pin+ po)2” if F(n) = n22", and the form n?(pon* + pin + po)3” 
if F(n) = (n? + 1)3". @ 


Care must be taken when s = | when solving recurrence relations of the type covered by 
Theorem 6. In particular, to apply this theorem with F(n) = byn; + by_-jny_-1 +--+: + bin + bo, 
the parameter s takes the value s = 1 (even though the term 1” does not explicitly appear). By 
the theorem, the form of the solution then depends on whether 1 is a root of the character- 
istic equation of the associated linear homogeneous recurrence relation. This is illustrated in 
Example 13, which shows how Theorem 6 can be used to find a formula for the sum of the first 
n positive integers. 


Let a, be the sum of the first n positive integers, so that 


n 
n= 2 k. 
k=1 
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Note that a, satisfies the linear nonhomogeneous recurrence relation 
An = 4y-1 +N. 


(To obtain a,, the sum of the first n positive integers, from a,—1, the sum of the first n — 1 
positive integers, we add n.) Note that the initial condition is a; = 1. 
The associated linear homogeneous recurrence relation for ay, is 


an = An-1.- 


The solutions of this homogeneous recurrence relation are given by a” =c(1)" =c, 


where c is a constant. To find all solutions of a, = a,—; +n, we need find only a single partic- 
ular solution. By Theorem 6, because F(n) =n =n - (1)” ands = 1 is a root of degree one of 
the characteristic equation of the associated linear homogeneous recurrence relation, there is a 
particular solution of the form n(pin + po) = pin? + pon. 

Inserting this into the recurrence relation gives pjn* + pon = pi(n—1)?+ 
po(n—1)+ n. Simplifying, we see that n(2pi —1)+(po-— pi) =0, which means 
that 2p; — 1 = O and po — p; = 0, so po = p} = 1/2. Hence, 


2 
(p) __ 1 n_ n(n+1) 
m=atZ= 9 


is a particular solution. Hence, all solutions of the original recurrence relation a, = a,_, +n are 


given by a, = a +a?) =c+n(n+ 1)/2.Becausea; = 1, wehavel =a) =c+1-2/2= 


c+1,soc = 0. It follows that a, = n(n + 1)/2. (This is the same formula given in Table 2 in 
Section 2.4 and derived previously.) < 


1. Determine which of these are linear homogeneous recur- 4. Solve these recurrence relations together with the initial 


rence relations with constant coefficients. Also, find the 
degree of those that are. 

a) an = 3an—1 + 4an—2 + San—3 

b) an = 2nay—1 + an—2 C) dn = an—-1 + An—-4 

d) ay =an-1 +2 
f) an = an-2 


. Determine which of these are linear homogeneous recur- 
rence relations with constant coefficients. Also, find the 
degree of those that are. 

b) a, =3 


a) an = 3an_2 
; d) ay = an—1 + 2an-3 


©) =a), 
€) dy =ay-1/Nn 
£) dy =4y-1 + Q—-2 +n+3 

2) Gn = 4ay—2 + San—4 + 9Gn—7 

. Solve these recurrence relations together with the initial 
conditions given. 


a) dy) = 2a,_; forn > 1,ag =3 


conditions given. 

a) dn = dn—1 + 6ay_2 for n > 2, a9 = 3, a; = 6 
b) dy = Tan—1 — 10an—2 forn > 2, a9 = 2,a; = 1 
C) ay = 6ay_1 — 8ay_2 forn > 2, a9 = 4, ay = 10 


e) an = ae + an—2 d) ay = 2dy_-1 — An—2 forn > 2,a9 = 4, a, = 1 
2) dn =GQr-1 +n €) ay = an_-2 forn > 2,a9 =5,a; = —1 
f) a, = —6ay,_1 — 9ay—2 forn > 2, ag = 3, a, = —-3 


2) dn42 = —4an41 + Say for n > 0, ag = 2, a1 = 8 


. How many different messages can be transmitted in n mi- 


croseconds using the two signals described in Exercise 19 
in Section 8.1? 


. How many different messages can be transmitted in n 


microseconds using three different signals if one signal 
requires 1 microsecond for transmittal, the other two sig- 
nals require 2 microseconds each for transmittal, and a 
signal in a message is followed immediately by the next 
signal? 


b) ad, =ay,_-) forn > 1,a9 = 2 7. In how many ways can a2 x n rectangular checkerboard 
C) dn = 5an_—| — 6dn_—2 forn > 2,a9 = 1, a; = 0 be tiled using 1 x 2 and 2 x 2 pieces? 

d) dn = 4an—-1 — 4ayn—2 for n > 2, a9 = 6, a; = 8 8. A model for the number of lobsters caught per year is 
€) dy = —4an-1 — 4an—-2 forn > 2,a9 = 0, a) = 1 based on the assumption that the number of lobsters 


f) a, = 4ay_2 forn > 2,a9 = 0,a, =4 
2) Qn = 4n-2/4 forn > 2,a9 = 1,a, = 0 


caught in a year is the average of the number caught in 
the two previous years. 


9. 


*10. 
11. 


12. 


13. 


14. 


15. 


*16. 
17. 


18. 


19. 


20. 


21. 


22. 


a) Find a recurrence relation for {L,}, where Ly, is the 
number of lobsters caught in year n, under the as- 
sumption for this model. 

b) Find L,, if 100,000 lobsters were caught in year | and 
300,000 were caught in year 2. 

A deposit of $100,000 is made to an investment fund at 

the beginning of a year. On the last day of each year two 

dividends are awarded. The first dividend is 20% of the 

amount in the account during that year. The second divi- 

dend is 45% of the amount in the account in the previous 

year. 

a) Find a recurrence relation for {P,}, where P, is the 
amount in the account at the end of n years if no money 
is ever withdrawn. 

b) How much is in the account after n years if no money 
has been withdrawn? 


Prove Theorem 2. 


The Lucas numbers satisfy the recurrence relation 
Ly = Ln-1 + Ln-2, 


and the initial conditions Lo = 2 and L; = 1. 

a) Show that Ly = fp-i+fno1 for n=2,3,..., 
where f;, is the nth Fibonacci number. 

b) Find an explicit formula for the Lucas numbers. 


Find the solution to ad, = 2d,—1| + dy—2 — 2ay_3 
forn = 3,4,5,..., with ag = 3, a; = 6, and az = 0. 


Find the solution to a, = 7d,—2 + 6a,—3 with ap = 9, 
a, = 10, and a2 = 32. 

Find the solution to a, = 5d,_2 — 4a,—4 with ag = 3, 
a, = 2, a2 = 6, and a3 = 8. 

Find the solution to dy = 2an—1 + 5an—2 — 6ay,—3 with 
ag = 7, a, = —4, and a2 = 8. 


Prove Theorem 3. 


Prove this identity relating the Fibonacci numbers and the 
binomial coefficients: 


fn41 = C0, 0) + Cin — 1,1) +---+ Ca —k,k), 
where n is a positive integer and k = |n /2]|. [Hint: Let 
a, = C(n,0)+ Cn — 1,1) +---+ C(n—k,k). Show 
that the sequence {a,} satisfies the same recurrence re- 


lation and initial conditions satisfied by the sequence of 
Fibonacci numbers. | 


Solve the recurrence relation a, = 6a,—1 — 12a,—-2 + 


8an—3 With ag = —5, a, = 4, and az = 88. 
Solve the recurrence relation ad, = —3dy,_—1 — 3dy—2 
dy—3 With ag = 5, ay = —9, and az = 15. 


Find the general form of the solutions of the recurrence 
relation dy = 8ay)—2 — 16ay_—4. 


What is the general form of the solutions of a linear homo- 
geneous recurrence relation if its characteristic equation 
has roots 1, 1, 1, 1, —2, —2, —2, 3, 3, —4? 


What is the general form of the solutions of a linear homo- 
geneous recurrence relation if its characteristic equation 
has the roots —1, —1, —1, 2,2,5,5,7? 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 
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Consider the nonhomogeneous linear recurrence relation 

Gn = 3an—| +2”. 

a) Show that a, = —2”+! isa solution of this recurrence 
relation. 


b) Use Theorem 5 to find all solutions of this recurrence 
relation. 

c) Find the solution with ap = 1. 

Consider the nonhomogeneous linear recurrence relation 

Ay = 2An—| +2”. 

a) Show that a, = n2” is a solution of this recurrence 
relation. 

b) Use Theorem 5 to find all solutions of this recurrence 
relation. 

c) Find the solution with ap = 2. 

a) Determine values of the constants A and B such 
that a, = An + B isa solution of recurrence relation 
dyn = 2a4n-1 +n +5. 

b) Use Theorem 5 to find all solutions of this recurrence 
relation. 

c) Find the solution of this recurrence relation with 
ag = 4. 


What is the general form of the particular so- 
lution guaranteed to exist by Theorem 6 of 
the linear nonhomogeneous recurrence relation 
Gn = 6dn—1 — 12dy—2 + 8ay_-3 + F(n) if 

a) F(n) =n?? b) F(n) = 2”? 

c) F(n) =n2"? d) F(n) = (—2)"? 

e) F(n) = n?2"9 f) F(n) =n3(—2)"? 

g) F(n) =3? 

What is the general form of the particular solution guaran- 


teed to exist by Theorem 6 of the linear nonhomogeneous 

recurrence relation dy, = 8dy,—2 — 16a,_4 + F(n) if 

a) F(n) =n°? b) F(n) = (-2)"? 

c) F(n) =n2"? d) F(n) =n?4"? 

e) F(n) = (n? — 2)(—2)"7 f) F(n) = n42"? 

g) F(n) =2? 

a) Find all solutions of the recurrence relation 
Gn = 2an—1 + 2n?. 

b) Find the solution of the recurrence relation in part (a) 
with initial condition a, = 4. 

a) Find all solutions of the recurrence relation 
an = 2an—1 + 3”. 

b) Find the solution of the recurrence relation in part (a) 
with initial condition a; = 5. 

a) Find all solutions of the recurrence relation a, = 
—Sayn—| — 6ay_2 + 42-4”. 

b) Find the solution of this recurrence relation with a; = 
56 and az = 278. 

Find all solutions of the recurrence relation a, = 

San—1 — 6dy—2 + 2” + 3n. [Hint: Look for a particular 

solution of the form gn2” + pin + po, where g, pi, and 

p2 are constants. ] 

Find the solution of the recurrence relation a, = 

2ayn_-1 +3-+2”. 

Find all solutions of the recurrence relation a, = 

4an—1 — 4an—2 + (n + 1)2”. 
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34. 


35. 


36. 


37. 


38. 


* 39. 


* 40. 


*41, 


42. 


43. 


Find all solutions of the recurrence relation a, = 
Tayn—| — 16an—2 + 12ay,_3 + n4" with ao = —2, 
ay = 0, and az = 5: 


Find the solution of the recurrence relation a, = 
4dyn—| — 3dn_2 +2” +n +3 with ag = 1 anda, = 4. 


Let a, be the sum of the first n perfect squares, that 
is, dy = eke: Show that the sequence {a,} sat- 
isfies the linear nonhomogeneous recurrence relation 
Gn = An_1 +n” and the initial condition aj = 1. Use 
Theorem 6 to determine a formula for a, by solving this 
recurrence relation. 


Let a, be the sum of the first n triangular numbers, that is, 
an = oe te, Where t, = k(k + 1)/2. Show that {ay} 
satisfies the linear nonhomogeneous recurrence relation 
An = An—, +n(n + 1)/2 and the initial conditiona; = 1. 
Use Theorem 6 to determine a formula for a, by solving 
this recurrence relation. 


a) Find the characteristic roots of the linear homo- 
geneous recurrence relation a, = 2d,—1 — 2dy,_2. 
[Note: These are complex numbers. ] 


b) Find the solution of the recurrence relation in part (a) 
with dg = 1 and a; = 2. 


a) Find the characteristic roots of the linear homoge- 
neous recurrence relation a, = a,_4. [Note: These 
include complex numbers. ] 


b) Find the solution of the recurrence relation in part (a) 
with ag = 1, a; = 0, a2 = —1,anda3 = 1. 


Solve the simultaneous recurrence relations 
An = 3dn—1 + 2bn-1 
bn = An—1 + 2bn-1 

with ap = 1 and bo = 2. 


a) Use the formula found in Example 4 for f,,, the nth 
Fibonacci number, to show that f, is the integer 
closest to 


1 1 4 is n 
V5\ 2 J 
b) Determine for which n f, is greater than 
I 1 4 J5 n 
J5 2 


and for which n f, is less than 
VATEMSY: 
JEN 2 fe 


Show that if a, = a,—-1 +ay_-2, a9 =s and a; =f, 
where s and ¢ are constants, then a, = sf,_1 + tfn for 
all positive integers n. 


Express the solution of the linear nonhomogenous 
recurrence relation dy, =dn,—; +d,_2+1 for n>2 


* 44, 


45. 


46. 


where ag = 0 and a; = | in terms of the Fibonacci num- 
bers. [Hint: Let b, = a, + 1 and apply Exercise 42 to the 
sequence b,.] 


(Linear algebra required) Let A, be the n x n matrix 
with 2s on its main diagonal, 1s in all positions next to a 
diagonal element, and Os everywhere else. Find a recur- 
rence relation for d,, the determinant of A,,. Solve this 
recurrence relation to find a formula for d,. 


Suppose that each pair of a genetically engineered species 
of rabbits left on an island produces two new pairs of rab- 
bits at the age of 1 month and six new pairs of rabbits at 
the age of 2 months and every month afterward. None of 
the rabbits ever die or leave the island. 


a) Find a recurrence relation for the number of pairs of 
rabbits on the island n months after one newborn pair 
is left on the island. 

b) By solving the recurrence relation in (a) determine 
the number of pairs of rabbits on the island n months 
after one pair is left on the island. 


Suppose that there are two goats on an island initially. 
The number of goats on the island doubles every year by 
natural reproduction, and some goats are either added or 
removed each year. 


a) Construct a recurrence relation for the number of 
goats on the island at the start of the nth year, as- 
suming that during each year an extra 100 goats are 
put on the island. 

b) Solve the recurrence relation from part (a) to find the 
number of goats on the island at the start of the nth 
year. 

c) Construct a recurrence relation for the number of 
goats on the island at the start of the nth year, as- 
suming that n goats are removed during the nth year 
for eachn > 3. 

d) Solve the recurrence relation in part (c) for the number 
of goats on the island at the start of the nth year. 


47. A new employee at an exciting new software company 


starts with a salary of $50,000 and is promised that at the 

end of each year her salary will be double her salary of 

the previous year, with an extra increment of $10,000 for 

each year she has been with the company. 

a) Construct a recurrence relation for her salary for her 
nth year of employment. 

b) Solve this recurrence relation to find her salary for her 
nth year of employment. 


Some linear recurrence relations that do not have constant co- 
efficients can be systematically solved. This is the case for 
recurrence relations of the form f(n)a, = g(n)adyn_-) + h(n). 
Exercises 48—50 illustrate this. 


* 48, 


a) Show that the recurrence relation 
f(M)an = g(N)an—} + h(n), 


for n > 1, and with ag = C, can be reduced to a re- 
currence relation of the form 


bn = bn-1 + Q()h(n), 


where b, = g(n+ 1)Q(n + Dap, with 


O(n) = (FD FOR)--- Ff — IN/(gMg@)---g@)). 


b) Use part (a) to solve the original recurrence relation 
to obtain 
_ C+ a1 POAO 
g(n+1l)O(n41) © 


*49, Use Exercise 48 to solve the recurrence relation 


(n+ lay = (n+ 3)an_1 +n, forn > 1, with ap = 1. 


50. It can be shown that C,,, the average number of com- 


parisons made by the quick sort algorithm (described in 
preamble to Exercise 50 in Section 5.4), when sorting n 
elements in random order, satisfies the recurrence relation 


8.3 Divide-and-Conquer Algorithms and Recurrence Relations 


#*51, 
*#* 52. 
53. 
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2 n—-1 
Ch=n+1+—=S 
n 
k=0 
forn = 1,2,..., with initial condition Cp = 0. 


a) Show that {C,,} also satisfies the recurrence relation 
nC, = (n+ 1)Cy_-1 + 2n forn = 1,2,.... 

b) Use Exercise 48 to solve the recurrence relation in 
part (a) to find an explicit formula for C,,. 

Prove Theorem 4. 

Prove Theorem 6. 

Solve the recurrence relation T (n) = nT2(n /2) with ini- 

tial condition T(1) = 6 when n = 2* for some inte- 

ger k. [Hint: Let n = 2* and then make the substitution 

ax = log T (2*) to obtain a linear nonhomogeneous re- 

currence relation. ] 


Divide-and-Conquer Algorithms and Recurrence Relations 


Introduction 


Many recursive algorithms take a problem with a given input and divide it into one or more 
smaller problems. This reduction is successively applied until the solutions of the smaller prob- 
lems can be found quickly. For instance, we perform a binary search by reducing the search for 
an element in a list to the search for this element in a list half as long. We successively apply 
this reduction until one element is left. When we sort a list of integers using the merge sort, we 
split the list into two halves of equal size and sort each half separately. We then merge the two 
sorted halves. Another example of this type of recursive algorithm is a procedure for multiplying 
integers that reduces the problem of the multiplication of two integers to three multiplications 
of pairs of integers with half as many bits. This reduction is successively applied until integers 
with one bit are obtained. These procedures follow an important algorithmic paradigm known 
as divide-and-conquer, and are called divide-and-conquer algorithms, because they divide 
a problem into one or more instances of the same problem of smaller size and they conquer 
the problem by using the solutions of the smaller problems to find a solution of the original 
problem, perhaps with some additional work. 

In this section we will show how recurrence relations can be used to analyze the compu- 
tational complexity of divide-and-conquer algorithms. We will use these recurrence relations 
to estimate the number of operations used by many different divide-and-conquer algorithms, 
including several that we introduce in this section. 


Links 


“Divide et impera” 
(translation: “Divide and 
conquer” - Julius Caesar 


Divide-and-Conquer Recurrence Relations 


Suppose that a recursive algorithm divides a problem of size n into a subproblems, where each 
subproblem is of size n/b (for simplicity, assume that 7 is a multiple of b; in reality, the smaller 
problems are often of size equal to the nearest integers either less than or equal to, or greater 
than or equal to, n/b). Also, suppose that a total of g(n) extra operations are required in the 
conquer step of the algorithm to combine the solutions of the subproblems into a solution of 
the original problem. Then, if f(m) represents the number of operations required to solve the 
problem of size n, it follows that f satisfies the recurrence relation 


f(n) = af (n/b) + gin). 


This is called a divide-and-conquer recurrence relation. 
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EXAMPLE 1 


Extra p> 
Examples i 


EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


Links 


We will first set up the divide-and-conquer recurrence relations that can be used to study 
the complexity of some important algorithms. Then we will show how to use these divide-and- 
conquer recurrence relations to estimate the complexity of these algorithms. 


Binary Search We introduced a binary search algorithm in Section 3.1. This binary search 
algorithm reduces the search for an element in a search sequence of size n to the binary search 
for this element in a search sequence of size n/2, when n is even. (Hence, the problem of size n 
has been reduced to one problem of size n/2.) Two comparisons are needed to implement this 
reduction (one to determine which half of the list to use and the other to determine whether any 
terms of the list remain). Hence, if f() is the number of comparisons required to search for an 
element in a search sequence of size n, then 


f(n) = f(n/2) +2 


when n is even. < 


Finding the Maximum and Minimum of a Sequence Consider the following algorithm for 
locating the maximum and minimum elements of a sequence qd), a2, ..., dn. Ifn = 1, then a, is 
the maximum and the minimum. If n > 1, split the sequence into two sequences, either where 
both have the same number of elements or where one of the sequences has one more element 
than the other. The problem is reduced to finding the maximum and minimum of each of the 
two smaller sequences. The solution to the original problem results from the comparison of the 
separate maxima and minima of the two smaller sequences to obtain the overall maximum and 
minimum. 

Let f(n) be the total number of comparisons needed to find the maximum and minimum 
elements of the sequence with n elements. We have shown that a problem of size n can be 
reduced into two problems of size n/2, when n is even, using two comparisons, one to compare 
the maxima of the two sequences and the other to compare the minima of the two sequences. 
This gives the recurrence relation 


f(n) = 2f(n/2) +2 


when n is even. < 


Merge Sort The merge sort algorithm (introduced in Section 5.4) splits a list to be sorted 
with n items, where n is even, into two lists with n/2 elements each, and uses fewer than n 
comparisons to merge the two sorted lists of n/2 items each into one sorted list. Consequently, 
the number of comparisons used by the merge sort to sort a list of n elements is less than M(n), 
where the function M(n) satisfies the divide-and-conquer recurrence relation 


M(n) = 2M (n/2) +n. < 


Fast Multiplication of Integers Surprisingly, there are more efficient algorithms than the con- 
ventional algorithm (described in Section 4.2) for multiplying integers. One of these algorithms, 
which uses a divide-and-conquer technique, will be described here. This fast multiplication al- 
gorithm proceeds by splitting each of two 2n-bit integers into two blocks, each with n bits. 
Then, the original multiplication is reduced from the multiplication of two 2n-bit integers to 
three multiplications of n-bit integers, plus shifts and additions. 

Suppose that a and b are integers with binary expansions of length 2n (add initial bits of 
zero in these expansions if necessary to make them the same length). Let 


@ = (42n—1A2n~2 «+ + a\aQ)2 and b = (b2n—1b2n—-2 +++ b,bo)2. 


EXAMPLE 5 


Links 
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Let 
a=2"A,;+Ao0, b=2"B, + Bo, 
where 


Al = (@an-1°** Gn414n)2, Ao = (an-1-+ + @140)2, 
By = (bon-1 +++ bn4ibn)2, Bo = (bn-1-- + 51 b0)2. 


The algorithm for fast multiplication of integers is based on the fact that ab can be 
rewritten as 


ab = (2°" + 2")A,By + 2"(Ay — Ao)(Bo — By) + (2" + 1)AoBo. 


The important fact about this identity is that it shows that the multiplication of two 2n-bit 
integers can be carried out using three multiplications of n-bit integers, together with additions, 
subtractions, and shifts. This shows that if f(n) is the total number of bit operations needed to 
multiply two n-bit integers, then 


fn) =3f(n)+ Cn. 


The reasoning behind this equation is as follows. The three multiplications of n-bit integers are 
carried out using 3 f(7)-bit operations. Each of the additions, subtractions, and shifts uses a 
constant multiple of n-bit operations, and Cn represents the total number of bit operations used 
by these operations. < 


Fast Matrix Multiplication In Example 7 of Section 3.3 we showed that multiplying two 
n Xn matrices using the definition of matrix multiplication required n> multiplications and 
n?(n — 1) additions. Consequently, computing the product of two n x n matrices in this way 
requires O(n*) operations (multiplications and additions). Surprisingly, there are more efficient 
divide-and-conquer algorithms for multiplying twon x n matrices. Such an algorithm, invented 
by Volker Strassen in 1969, reduces the multiplication of twon x n matrices, when n is even, to 
seven multiplications of two (n/2) x (n/2) matrices and 15 additions of (n/2) x (n/2) matrices. 
(See [CoLeRiSt09] for the details of this algorithm.) Hence, if f (7) is the number of operations 
(multiplications and additions) used, it follows that 


f (n) = 7f (n/2) + 15n7/4 
when nis even. < 


As Examples 1-5 show, recurrence relations of the form f(n) = af (n/b) + g(n) arise in 
many different situations. It is possible to derive estimates of the size of functions that satisfy 
such recurrence relations. Suppose that f satisfies this recurrence relation whenever n is divisible 
by b. Let n = b*, where k is a positive integer. Then 

f() = af (n/b) + g(n) 

= a°f (n/b°) + ag(n/b) + g(n) 


= a°f (n/b*) + a?g(n/b’) + ag(n/b) + g(n) 


k-1 


= a‘f (n/b‘) + > a! g(n/b!). 


j=0 
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Because n/b* = 1, it follows that 


k-1 
fn) =a‘ fl) + So a/g(n/b’). 


j=0 


We can use this equation for f (7) to estimate the size of functions that satisfy divide-and-conquer 
relations. 


THEOREM 1 Let f be an increasing function that satisfies the recurrence relation 
f(n) =af(n/b) +c 


whenever n is divisible by b, where a > 1, b is an integer greater than 1, and c is a positive 
real number. Then 


_ | O(n8>%) if a > 1, 
f(n) is ; 
O(logn) ifa=1. 
Furthermore, when n = b‘and a £ 1, where k is a positive integer, 


f(n) = Cyn!84 + Co, 


where C; = f(1) + c/(a — 1) and Co = —c/(a — 1). 


Proof: First letn = b*. From the expression for f(n) obtained in the discussion preceding the 
® theorem, with g(n) = c, we have 


k-1 k-1 
fa) =a‘ f(l)+ Do aic=a'fj)+e) a’. 
j=0 j=0 


When a = 1 we have 


f(n) = fC) + ck. 


Because n = b‘, we have k = log, n. Hence, 
f(n) = f() +elogyn. 


When n is not a power of b, we have b* <n < b‘+!, for a positive integer k. Because f is 
increasing, it follows that f(n) < fot!) =f)+ck+)=(fd)+tot+ck< (f+ 
c) + clog, n. Therefore, in both cases, f(n) is O(logn) when a = 1. 

Now suppose that a > 1. First assume that n = b*, where k is a positive integer. From the 
formula for the sum of terms of a geometric progression (Theorem | in Section 2.4), it follows 
that 


f(n) =a*f (1) +c(ak — 1)/(a—- 1) 
=a‘[f(1)+c/(a— 1] —c/(a-V 
= Cyn'84 + Cp, 


EXAMPLE 6 


Extra & 
Examples Qi 


EXAMPLE 7 


EXAMPLE 8 


8.3 Divide-and-Conquer Algorithms and Recurrence Relations 531 


because a* = q!©8" = n!°o4 (see Exercise 4 in Appendix 2), where C} = f(1) + c/(a— 1) 
and C2 = —c/(a — 1). 
Now suppose that n is not a power of b. Then bk <n < bk+!, where k is a nonnegative 
integer. Because f is increasing, 
fn) s fO") =Cya" +0) 
< (Cya)al8" + Cy 
= (Cya)n'°%4 + Cp, 


because k < log,n <k +1. 
Hence, we have f(n) is O(n'°%), <j 


Examples 6—9 illustrate how Theorem | is used. 


Let f(n) = 5f(n/2) +3 and f(1) = 7. Find FC"), where k is a positive integer. Also, estimate 
f(n) if f is an increasing function. 


Solution: From the proof of Theorem 1, with a = 5, b = 2, and c = 3, we see that if n = oF 
then 


f(n) =a*[f(1) +c/@— 1] +[-c/@-)] 
= 5‘17 + (3/4)] — 3/4 
= 5§(31/4) — 3/4. 


Also, if f(n) is increasing, Theorem 1 shows that f(n) is O(n!°&¢) = O(n!85), < 


We can use Theorem | to estimate the computational complexity of the binary search 
algorithm and the algorithm given in Example 2 for locating the minimum and maximum of a 
sequence. 


Give a big-O estimate for the number of comparisons used by a binary search. 


Solution: In Example 1 it was shown that f(n) = f(n/2) + 2 when n is even, where f is the 
number of comparisons required to perform a binary search on a sequence of size n. Hence, 
from Theorem 1, it follows that f(n) is O(logn). < 


Give a big-O estimate for the number of comparisons used to locate the maximum and minimum 
elements in a sequence using the algorithm given in Example 2. 


Solution: In Example 2 we showed that f(n) = 2 f(n/2) + 2, when n is even, where f is the 
number of comparisons needed by this algorithm. Hence, from Theorem |, it follows that f(n) 
is O(n'°82) = O(n). 


We now state a more general, and more complicated, theorem, which has Theorem | as 
a special case. This theorem (or more powerful versions, including big-Theta estimates) is 
sometimes known as the master theorem because it is useful in analyzing the complexity of 
many important divide-and-conquer algorithms. 
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THEOREM 2 


EXAMPLE 9 


EXAMPLE 10 


EXAMPLE 11 


EXAMPLE 12 


Links 


MASTER THEOREM Let f bean increasing function that satisfies the recurrence relation 
f (n) = af (n/b) + cn 


whenever n = b* , where k is a positive integer, a > 1, b is an integer greater than 1, and c 
and d are real numbers with c positive and d nonnegative. Then 


O(n“) ite =>" 
f(n) is { O(n? logn) ifa= ae 
O(n'$8*) ifa > b?. 


The proof of Theorem 2 is left for the reader as Exercises 29-33. 


Complexity of Merge Sort In Example 3 we explained that the number of comparisons used 
by the merge sort to sort a list of n elements is less than M(n), where M(n) = 2M(n/2) +n. 
By the master theorem (Theorem 2) we find that M(n) is O(n logn), which agrees with the 
estimate found in Section 5.4. < 


Give a big-O estimate for the number of bit operations needed to multiply two n-bit integers 
using the fast multiplication algorithm described in Example 4. 


Solution: Example 4 shows that f(n) = 3 f(n/2) + Cn, when n is even, where f (7) is the 
number of bit operations required to multiply two n-bit integers using the fast multiplication 
algorithm. Hence, from the master theorem (Theorem 2), it follows that f(m) is O(n’83), 
Note that log 3 ~ 1.6. Because the conventional algorithm for multiplication uses O(n”) bit 
operations, the fast multiplication algorithm is a substantial improvement over the conventional 
algorithm in terms of time complexity for sufficiently large integers, including large integers 
that occur in practical applications. 


Give a big-O estimate for the number of multiplications and additions required to multiply two 
n Xn matrices using the matrix multiplication algorithm referred to in Example 5. 


Solution: Let f(n) denote the number of additions and multiplications used by the algorithm 
mentioned in Example 5 to multiply two n x n matrices. We have f (n) = 7 f (n/2) + 15n?/4, 
when n is even. Hence, from the master theorem (Theorem 2), it follows that f (1) is O (n!87), 
Note that log 7 ~ 2.8. Because the conventional algorithm for multiplying two n x n matrices 
uses O(n?) additions and multiplications, it follows that for sufficiently large integers n, includ- 
ing those that occur in many practical applications, this algorithm is substantially more efficient 
in time complexity than the conventional algorithm. < 


THE CLOSEST-PAIR PROBLEM We conclude this section by introducing a divide-and- 
conquer algorithm from computational geometry, the part of discrete mathematics devoted to 
algorithms that solve geometric problems. 


The Closest-Pair Problem Consider the problem of determining the closest pair of points 
in a set of n points (x1, y1),.--, (4n, Yn) in the plane, where the distance between two points 


(x;, yi) and (x;, yj) is the usual Euclidean distance | _ ai? + (Oi - irs This problem 
arises in many applications such as determining the closest pair of airplanes in the air space at a 
particular altitude being managed by an air traffic controller. How can this closest pair of points 
be found in an efficient way? 


It took researchers more 
than 10 year to find an 
algorithm with O(n log n) 
complexity that locates 
the closest pair of points 
among n points. 
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In this illustration the problem of finding the 
d, | closest pair in a set of 16 points is reduced to 
dr two problems of finding the closest pair in 
| @ | a set of eight points and the problem of 
closest determining whether there are points closer 
. pair than d = min(d;,, dr) within the strip of 
width 2d centered at €. 
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FIGURE 1 The Recursive Step of the Algorithm for Solving the Closest-Pair Problem. 


Solution: To solve this problem we can first determine the distance between every pair of 
points and then find the smallest of these distances. However, this approach requires O(n?) 
computations of distances and comparisons because there are C(n, 2) = n(n — 1)/2 pairs of 
points. Surprisingly, there is an elegant divide-and-conquer algorithm that can solve the closest- 
pair problem for n points using O(n logn) computations of distances and comparisons. The 
algorithm we describe here is due to Michael Samos (see [PrSa85]). 

For simplicity, we assume that n = 2‘, where k is a positive integer. (We avoid some 
technical considerations that are needed when n is not a power of 2.) When n = 2, we have only 
one pair of points; the distance between these two points is the minimum distance. At the start 
of the algorithm we use the merge sort twice, once to sort the points in order of increasing x 
coordinates, and once to sort the points in order of increasing y coordinates. Each of these sorts 
requires O(n logn) operations. We will use these sorted lists in each recursive step. 

The recursive part of the algorithm divides the problem into two subproblems, each involving 
half as many points. Using the sorted list of the points by their x coordinates, we construct a 
vertical line @ dividing the n points into two parts, a left part and a right part of equal size, each 
containing n/2 points, as shown in Figure 1. (If any points fall on the dividing line £, we divide 
them among the two parts if necessary.) At subsequent steps of the recursion we need not sort 
on x coordinates again, because we can select the corresponding sorted subset of all the points. 
This selection is a task that can be done with O(n) comparisons. 

There are three possibilities concerning the positions of the closest points: (1) they are both 
in the left region L, (2) they are both in the right region R, or (3) one point is in the left region 
and the other is in the right region. Apply the algorithm recursively to compute dy and dr, 
where dy is the minimum distance between points in the left region and dp is the minimum 
distance between points in the right region. Let d = min(dz, dr). To successfully divide the 
problem of finding the closest two points in the original set into the two problems of finding the 
shortest distances between points in the two regions separately, we have to handle the conquer 
part of the algorithm, which requires that we consider the case where the closest points lie in 
different regions, that is, one point is in L and the other in R. Because there is a pair of points 
at distance d where both points lie in R or both points lie in L, for the closest points to lie in 
different regions requires that they must be a distance less than d apart. 

For a point in the left region and a point in the right region to lie at a distance less than d apart, 
these points must lie in the vertical strip of width 2d that has the line ¢ as its center. (Otherwise, 
the distance between these points is greater than the difference in their x coordinates, which 
exceeds d.) To examine the points within this strip, we sort the points so that they are listed in 
order of increasing y coordinates, using the sorted list of the points by their y coordinates. At 
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£ 
d/2 d/2 d/2 d/2 
d/2 
At most eight points, including p, 
can lie in or on the 2d X d rectangle 
centered at € because at most one 
point can lie in or on each of the 
d/2 eight (d/2) X (d/2) squares. 
d/|2 
° 
Pp 


FIGURE 2 Showing That There Are at Most Seven Other Points to Consider for Each 
Point in the Strip. 


each recursive step, we form a subset of the points in the region sorted by their y coordinates 
from the already sorted set of all points sorted by their y coordinates, which can be done 
with O(n) comparisons. 

Beginning with a point in the strip with the smallest y coordinate, we successively examine 
each point in the strip, computing the distance between this point and all other points in the strip 
that have larger y coordinates that could lie at a distance less than d from this point. Note that 
to examine a point p, we need only consider the distances between p and points in the set that 
lie within the rectangle of height d and width 2d with p on its base and with vertical sides at 
distance d from £. 

We can show that there are at most eight points from the set, including p, in or on this 2d x d 
rectangle. To see this, note that there can be at most one point in each of the eight d/2 x d/2 
squares shown in Figure 2. This follows because the farthest apart points can be on or within 

© one of these squares is the diagonal length d/./2 (which can be found using the Pythagorean 
theorem), which is less than d, and each of these d/2 x d/2 squares lies entirely within the left 
region or the right region. This means that at this stage we need only compare at most seven 
distances, the distances between p and the seven or fewer other points in or on the rectangle, 
with d. 

Because the total number of points in the strip of width 2d does not exceed n (the total 
number of points in the set), at most 7n distances need to be compared with d to find the 
minimum distance between points. That is, there are only 7n possible distances that could be 
less than d. Consequently, once the merge sort has been used to sort the pairs according to their 
x coordinates and according to their y coordinates, we find that the increasing function f (n) 
satisfying the recurrence relation 


f(n) = 2f(n/2) + 7n, 


where f (2) = 1, exceeds the number of comparisons needed to solve the closest-pair problem 
forn points. By the master theorem (Theorem 2), it follows that f (1) is O(n log n). The two sorts 
of points by their x coordinates and by their y coordinates each can be done using O(n logn) 
comparisons, by using the merge sort, and the sorted subsets of these coordinates at each of the 
O(logn) steps of the algorithm can be done using O(n) comparisons each. Thus, we find that 
the closest-pair problem can be solved using O(n logn) comparisons. < 


Exercises 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


. How many comparisons are needed for a binary search 


in a set of 64 elements? 


. How many comparisons are needed to locate the max- 


imum and minimum elements in a sequence with 128 
elements using the algorithm in Example 2? 


- Multiply (1110)2 and (1010) using the fast multiplica- 


tion algorithm. 


. Express the fast multiplication algorithm in pseudocode. 


. Determine a value for the constant C in Example 4 and 


use it to estimate the number of bit operations needed to 
multiply two 64-bit integers using the fast multiplication 
algorithm. 


. How many operations are needed to multiply two 32 x 32 


matrices using the algorithm referred to in Example 5? 


. Suppose that f(n) = f(n/3) + 1 when n is a positive 


integer divisible by 3, and f(1) = 1. Find 
a) f(3). b) f(27). c) f (729). 


. Suppose that f(n) = 2 f (n/2) + 3 whenn is an even pos- 


itive integer, and f(1) = 5. Find 


a) f(2). b) f(8). c) f(64). d) f (1024). 


. Suppose that f(n) = f(n/5) + 3n? when n is a positive 


integer divisible by 5, and f(1) = 4. Find 
a) f(5). b) f (125). ce) f (3125). 


Find f (2) whenn = 2*, where f satisfies the recurrence 
relation f(n) = f(n/2) + 1 with f(1) = 1. 


Give a big-O estimate for the function f in Exercise 10 
if f is an increasing function. 


Find f (n) when n = 3‘, where f satisfies the recurrence 
relation f(n) = 2f(n/3) +4 with f(1) = 1. 


Give a big-O estimate for the function f in Exercise 12 
if f is an increasing function. 


Suppose that there are n = 2* teams in an elimination 
tournament, where there are n/2 games in the first round, 
with the n/2 = 2‘! winners playing in the second round, 
and so on. Develop a recurrence relation for the number 
of rounds in the tournament. 


How many rounds are in the elimination tournament de- 
scribed in Exercise 14 when there are 32 teams? 


Solve the recurrence relation for the number of rounds in 
the tournament described in Exercise 14. 


Suppose that the votes of n people for different candi- 

dates (where there can be more than two candidates) for 

a particular office are the elements of a sequence. A per- 

son wins the election if this person receives a majority of 

the votes. 

a) Devise a divide-and-conquer algorithm that deter- 
mines whether a candidate received a majority and, 
if so, determine who this candidate is. [Hint: Assume 
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18. 


19. 


20. 


21. 


22. 
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that n is even and split the sequence of votes into 
two sequences, each with n/2 elements. Note that a 
candidate could not have received a majority of votes 
without receiving a majority of votes in at least one 
of the two halves. ] 


b) Use the master theorem to give a big-O estimate for 
the number of comparisons needed by the algorithm 
you devised in part (a). 

Suppose that each person in a group of n people votes for 

exactly two people from a slate of candidates to fill two 

positions on a committee. The top two finishers both win 
positions as long as each receives more than n/2 votes. 

a) Devise a divide-and-conquer algorithm that deter- 
mines whether the two candidates who received the 
most votes each received at least n/2 votes and, if so, 
determine who these two candidates are. 


b) Use the master theorem to give a big-O estimate for 
the number of comparisons needed by the algorithm 
you devised in part (a). 

a) Set up a divide-and-conquer recurrence relation 
for the number of multiplications required to 
compute x”, where x is a real number and n is a 
positive integer, using the recursive algorithm from 
Exercise 26 in Section 5.4. 


b) Use the recurrence relation you found in part (a) to 
construct a big-O estimate for the number of mul- 
tiplications used to compute x” using the recursive 
algorithm. 


a) Set up a divide-and-conquer recurrence relation for 
the number of modular multiplications required to 
compute a” mod m, where a, m, and n are pos- 
itive integers, using the recursive algorithms from 
Example 4 in Section 5.4. 


b) Use the recurrence relation you found in part (a) to 
construct a big-O estimate for the number of modular 
multiplications used to compute a” mod m using the 
recursive algorithm. 


Suppose that the function f satisfies the recurrence rela- 

tion f(n) = 2 f(./n) + 1 whenever n is a perfect square 

greater than 1 and f(2) = 1. 

a) Find f(16). 

b) Give a big-O estimate for f (n). [Hint: Make the sub- 
stitution m = log n.] 

Suppose that the function f satisfies the recurrence re- 

lation f(n) = 2f(./n) + logn whenever n is a perfect 

square greater than | and f(2) = 1. 

a) Find f(16). 

b) Find a big-O estimate for f(n). [Hint: Make the sub- 
stitution m = log n.] 

This exercise deals with the problem of finding the largest 

sum of consecutive terms of a sequence of n real numbers. 

When all terms are positive, the sum of all terms provides 
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24, 


25. 


* 26. 


27. 


* 28. 


the answer, but the situation is more complicated when 

some terms are negative. For example, the maximum sum 

of consecutive terms of the sequence —2, 3, —1, 6, —7, 4 

is3 + (—1) + 6 = 8. (This exercise is based on [Be86].) 

Recall that in Exercise 56 in Section 8.1 we developed a 

dynamic programming algorithm for solving this prob- 

lem. Here, we first look at the brute-force algorithm 
for solving this problem; then we develop a divide-and- 
conquer algorithm for solving it. 

a) Use pseudocode to describe an algorithm that solves 
this problem by finding the sums of consecutive terms 
starting with the first term, the sums of consecutive 
terms starting with the second term, and so on, keep- 
ing track of the maximum sum found so far as the 
algorithm proceeds. 

b) Determine the computational complexity of the al- 
gorithm in part (a) in terms of the number of sums 
computed and the number of comparisons made. 

c) Devise a divide-and-conquer algorithm to solve this 
problem. [Hint: Assume that there are an even number 
of terms in the sequence and split the sequence into 
two halves. Explain how to handle the case when the 
maximum sum of consecutive terms includes terms in 
both halves. ] 

d) Use the algorithm from part (c) to find the maximum 
sum of consecutive terms of each of the sequences: 
—2,4,—-1,3,5,—6, 1,2; 4, 1,-3,7,-1,-5, 3, —2; 
and —1, 6,3, —4, —5, 8, —1, 7. 

e) Find a recurrence relation for the number of sums 
and comparisons used by the divide-and-conquer al- 
gorithm from part (c). 

f) Use the master theorem to estimate the computa- 
tional complexity of the divide-and-conquer algo- 
rithm. How does it compare in terms of computational 
complexity with the algorithm from part (a)? 

Apply the algorithm described in Example 12 for find- 

ing the closest pair of points, using the Euclidean dis- 

tance between points, to find the closest pair of the 

points (1,3), (1,7), (2,4), (2,9), (3, 1), G, 5), (4, 3), 

and (4, 7). 

Apply the algorithm described in Example 12 for finding 

the closest pair of points, using the Euclidean distance be- 

tween points, to find the closest pair of the points (1, 2), 

(1, 6), (2,4), (2, 8), B,D, G, 6), G, 10), (4, 3), (5, 1), 

(5,5), (5, 9), (6, 7), (7, 1), (7, 4), (7, 9), and (8, 6). 

Use pseudocode to describe the recursive algorithm for 

solving the closest-pair problem as described in Exam- 

ple 12. 

Construct a variation of the algorithm described in Ex- 

ample 12 along with justifications of the steps used by 

the algorithm to find the smallest distance between two 
points if the distance between two points is defined to be 
d((xj, yi), (Xj, Yj) = max(|x; — xj |, |yi — yjl)- 

Suppose someone picks a number x from a set of n 

numbers. A second person tries to guess the number 

by successively selecting subsets of the n numbers and 


asking the first person whether x is in each set. The 

first person answers either “yes” or “no.” When the first 

person answers each query truthfully, we can find x 

using log n queries by successively splitting the sets 

used in each query in half. Ulam’s problem, proposed by 

Stanislaw Ulam in 1976, asks for the number of queries 

required to find x, supposing that the first person is al- 

lowed to lie exactly once. 

a) Show that by asking each question twice, given anum- 
ber x and a set with n elements, and asking one more 
question when we find the lie, Ulam’s problem can be 
solved using 2 logn + 1 queries. 

b) Show that by dividing the initial set of n elements into 
four parts, each withn/4 elements, 1/4 of the elements 
can be eliminated using two queries. [Hint: Use two 
queries, where each of the queries asks whether the 
element is in the union of two of the subsets with n/4 
elements and where one of the subsets of n /4 elements 
is used in both queries. ] 

c) Show from part (b) that if f(m) equals the number 
of queries used to solve Ulam’s problem using the 
method from part (b) and n is divisible by 4, then 
f(a) = f(3n/4) +2. 

d) Solve the recurrence relation in part (c) for f (7). 

e) Is the naive way to solve Ulam’s problem by ask- 
ing each question twice or the divide-and-conquer 
method based on part (b) more efficient? The most 
efficient way to solve Ulam’s problem has been 
determined by A. Pelc [Pe87]. 


In Exercises 29-33, assume that f is an increasing function 
satisfying the recurrence relation f(n) = af (n/b) +cn4, 
where a > 1, b is an integer greater than 1, and c and d 
are positive real numbers. These exercises supply a proof of 
Theorem 2. 


* 29, 


30. 


*31. 


32. 


33. 


34. 


35. 


36. 


37. 


Show that if a = b4 and n is a power of b, then f(n) = 
f()n? + cn4 log, n. 

Use Exercise 29 to show that if a = b“, then fM@) is 
O(nt logn). 

Show that if a 4 b¢ and n is a power of b, then f(n) = 
Cin? + Con's, where Cy = b¢c/(b4 — a) and C2 = 
fl) + btc/(a — b¢). 

Use Exercise 31 to show that if a < b4, then f(n) is 
O(n?). 

Use Exercise 31 to show that if a > b“, then fM@) is 
O(n'®& a) 

Find f(n) whenn = 4*, where f satisfies the recurrence 
relation f(n) = 5f(n/4) + 6n, with f(1) = 1. 

Give a big-O estimate for the function f in Exercise 34 
if f is an increasing function. 


Find f(n) whenn = 2*, where f satisfies the recurrence 
relation f(n) = 8f (n/2) +n? with f (1) = 1. 


Give a big-O estimate for the function f in Exercise 36 
if f is an increasing function. 
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8.4 Generating Functions 


Links 


DEFINITION 1 


EXAMPLE 1 


Extra p> 
Examples 2 


EXAMPLE 2 


Introduction 


Generating functions are used to represent sequences efficiently by coding the terms of a se- 
quence as coefficients of powers of a variable x in a formal power series. Generating functions 
can be used to solve many types of counting problems, such as the number of ways to select 
or distribute objects of different kinds, subject to a variety of constraints, and the number of 
ways to make change for a dollar using coins of different denominations. Generating functions 
can be used to solve recurrence relations by translating a recurrence relation for the terms of 
a sequence into an equation involving a generating function. This equation can then be solved 
to find a closed form for the generating function. From this closed form, the coefficients of the 
power series for the generating function can be found, solving the original recurrence relation. 
Generating functions can also be used to prove combinatorial identities by taking advantage of 
relatively simple relationships between functions that can be translated into identities involving 
the terms of sequences. Generating functions are a helpful tool for studying many properties of 
sequences besides those described in this section, such as their use for establishing asymptotic 
formulae for the terms of a sequence. 
We begin with the definition of the generating function for a sequence. 


The generating function for the sequence dg, a,,..., Ax, ... Of real numbers is the infinite 
series 
Co 
G(x) = ao + aix +--+ + ayxk +++ = DO agxt. 
k=0 


Remark: The generating function for {a;} given in Definition | is sometimes called the ordinary 
generating function of {ax} to distinguish it from other types of generating functions for this 
sequence. 


The generating functions for the sequences {ax} with a, =3,a,=k+1, and qa = gk 
are ) 9 3x*, Deg (k + Ix*, and D2 9 2*x*, respectively. < 

We can define generating functions for finite sequences of real numbers by extending a 
finite sequence ag, a,,..., @, into an infinite sequence by setting a,4; = 0, dn42 = 0, and so 
on. The generating function G(x) of this infinite sequence {a,} is a polynomial of degree n 


because no terms of the form ajx! with j > n occur, that is, 


G(x) = agp tayx +--+ ayx". 


What is the generating function for the sequence 1, 1, 1, 1, 1, 1? 
Solution: The generating function of 1, 1, 1, 1, 1, 1 is 

14 eg ee et eee, 
By Theorem | of Section 2.4 we have 


(e® — 1)/@—-—D=14x%427 422 424742? 
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EXAMPLE 3 


EXAMPLE 4 


EXAMPLE 5 


THEOREM 1 


when x #1. Consequently, G(x) = (x® — 1)/(x — 1) is the generating function of the 
sequence 1, 1, 1, 1, 1, 1. [Because the powers of x are only place holders for the terms of 
the sequence in a generating function, we do not need to worry that G(1) is undefined. ] < 


Let m be a positive integer. Let a, = C(m, k), for k = 0, 1,2,...,m. What is the generating 
function for the sequence ag, a1, ..., Am? 
Solution: The generating function for this sequence is 

G(x) = C(m, 0) + C(m, 1)x + C(m, 2)x? +---+C(m,m)x". 


The binomial theorem shows that G(x) = (1 + x)’. < 


Useful Facts About Power Series 


When generating functions are used to solve counting problems, they are usually considered to 
be formal power series. Questions about the convergence of these series are ignored. However, 
to apply some results from calculus, it is sometimes important to consider for which x the 
power series converges. The fact that a function has a unique power series around x = 0 will 
also be important. Generally, however, we will not be concerned with questions of convergence 
or the uniqueness of power series in our discussions. Readers familiar with calculus can consult 
textbooks on this subject for details about power series, including the convergence of the series 
we consider here. 

We will now state some important facts about infinite series used when working with 
generating functions. A discussion of these and related results can be found in calculus texts. 


The function f(x) = 1/(1 — x) is the generating function of the sequence 1, 1,1, 1,..., be- 
cause 


(i= Hae tee a* 4a. 
for |x| < 1. < 


The function f(x) = 1/(1 — ax) is the generating function of the sequence 1, a,a”,a?,..., 
because 


1/d —ax)=1+ax+a7x7? +... 
when |ax| < 1, or equivalently, for |x| < 1/|a| fora 4 0. < 


We also will need some results on how to add and how to multiply two generating functions. 
Proofs of these results can be found in calculus texts. 


ety @i— sau adie ji— ben Then 


k 


fx)tae) = Dat bex* and f(xg(x) = So [| Yo ajbi_; | x“. 
k=0 j 


k=0 =0 
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Remark: Theorem 1 is valid only for power series that converge in an interval, as all series 
considered in this section do. However, the theory of generating functions is not limited to such 
series. In the case of series that do not converge, the statements in Theorem | can be taken as 
definitions of addition and multiplication of generating functions. 


We will illustrate how Theorem | can be used with Example 6. 


EXAMPLE6 Let f(x) =1/d—- x)*. Use Example 4 to find the coefficients ag, a1, a2, ... in the expansion 
{OS Fade. 


Solution: From Example 4 we see that 
Vie aie exr ha eee 
Hence, from Theorem 1, we have 
oo k oe) 


Yi=sP= >) | yo 1) = > be De, 


k=0 \ j=0 k=0 
J 4 


Remark: This result also can be derived from Example 4 by differentiation. Taking derivatives is 
a useful technique for producing new identities from existing identities for generating functions. 


To use generating functions to solve many important counting problems, we will need to 


apply the binomial theorem for exponents that are not positive integers. Before we state an 
extended version of the binomial theorem, we need to define extended binomial coefficients. 


DEFINITION 2 Let wu be a real number and k a nonnegative integer. Then the extended binomial coefficient 
({,) is defined by 


UN ihe Na ee it ke 0) 
Get thee = 0), 


EXAMPLE 7 | Find the values of the extended binomial coefficients (=) and ‘eae 


Solution: Taking u = —2 and k = 3 in Definition 2 gives us 


(2) _ 2-3-4) 


3 3! 


Similarly, taking u = 1/2 and k = 3 gives us 


1/2\ _ (1/2)(1//2 = 1I)1/2 = 2) 
( 3 ) 7 3! 


= (1/2)(—1/2)(—3/2)/6 
= 1/16. “ 
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Example 8 provides a useful formula for extended binomial coefficients when the top 
parameter is a negative integer. It will be useful in our subsequent discussions. 


EXAMPLE 8 _ When the top parameter is a negative integer, the extended binomial coefficient can be expressed 
in terms of an ordinary binomial coefficient. To see that this is the case, note that 


by definition of extended binomial coefficient 


(;") J eiiea= 1) een =re 1) 
PP r! 


_ aD alt + loon Geer = 1) 


factoring out —1 from each term in the numerator 


r! 
(-—1)’'(n+r—1)(n+r—2)---n ; ee 
= by the commutative law for multiplication 
r! 
(eDGae =I Siu 
— multiplying both the numerator and denominator 
ri(n— 1)! by (n—1)! 
(eral as ee , 
= (-1) by the definition of binomial coefficients 
r 
= (-1)'C(n+r-—1,r). using alternative notation for binomial 


coefficients 


We now state the extended binomial theorem. 


THEOREM 2 THE EXTENDED BINOMIAL THEOREM Let x be a real number with |x| < 1 and 
let uw be a real number. Then 


(> (‘) xk, 


Theorem 2 can be proved using the theory of Maclaurin series. We leave its proof to the reader 
with a familiarity with this part of calculus. 


Remark: When u is a positive integer, the extended binomial theorem reduces to the binomial 
theorem presented in Section 6.4, because in that case (i) =Oifk>u. 


Example 9 illustrates the use of Theorem 2 when the exponent is a negative integer. 


EXAMPLE 9 _ Find the generating functions for (1 + x)~” and (1 — x)~”, where n is a positive integer, using 
the extended binomial theorem. 


Solution: By the extended binomial theorem, it follows that 


‘al 4%)? _ > (;’) xk. 


k=0 


EXAMPLE 10 
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Using Example 8, which provides a simple formula for C): we obtain 


+x" = S0CbD*cn+k-1, bx. 
k=0 


Replacing x by —.x, we find that 


(l-x) "= > Cintk—-1,k)x*. 
k=0 < 


Table | presents a useful summary of some generating functions that arise frequently. 


Remark: Note that the second and third formulae in this table can be deduced from the first 
formula by substituting ax and x” for x, respectively. Similarly, the sixth and seventh formulae 
can be deduced from the fifth formula using the same substitutions. The tenth and eleventh can 
be deduced from the ninth formula by substituting —x and ax for x, respectively. Also, some 
of the formulae in this table can be derived from other formulae using methods from calculus 
(such as differentiation and integration). Students are encouraged to know the core formulae in 
this table (that is, formulae from which the others can be derived, perhaps the first, fourth, fifth, 
eighth, ninth, twelfth, and thirteenth formulae) and understand how to derive the other formulae 
from these core formulae. 


Counting Problems and Generating Functions 

Generating functions can be used to solve a wide variety of counting problems. In particular, they 
can be used to count the number of combinations of various types. In Chapter 6 we developed 
techniques to count the r-combinations from a set with n elements when repetition is allowed 


and additional constraints may exist. Such problems are equivalent to counting the solutions to 
equations of the form 


ep teat---+te, =C, 


where C is a constant and each e; is a nonnegative integer that may be subject to a specified 
constraint. Generating functions can also be used to solve counting problems of this type, as 
Examples 10-12 show. 


Find the number of solutions of 
ey tea+e3= 17, 


where e1, e2, and e3 are nonnegative integers with 2 < ej < 5,3 < e2 < 6,and4 < e3 <7. 


Solution: The number of solutions with the indicated constraints is the coefficient of x!” in the 
expansion of 


(x? + x? + x? x°)(x? + x6 4x04 x°)(x4 + x +x x’), 


542 8/ Advanced Counting Techniques 


TABLE 1 Useful Generating Functions. 


+x)" = 52 Cin, bx 
k=0 


=14+C(n, 1I)x + C(n, 2)x7 +++. +x" 


n 
(l+ax)" = a C(n, dak x* 
k=0 
=14+C(a, lax + C(n, 2)a?x? eee 


n 
tae = 2s Co. bs" C(n, k/r) if r | k; 0 otherwise 
k=0 
=14+C(n, 1)x" + Cl, 2)x¥ +--+ x™ 


n 
= So xtaltxtar te 4x" 1 if k <n; 0 otherwise 
k=0 


[o.e) 
=\Soxtaltxtars-. 
k=0 


oe) 
= Yoatxt =1+4+ax+ax? +... 
k=0 


lif r | k; 0 otherwise 


Cnt+¢k—1,k)=Cin+k—1,n—1) 
(-1§ Can +k—1,k) = (-1l)*!C+k—1,n-1) 


[o.e) 
=>) Ca+k—-1,katx* Cntk—1,)ak =C(n+k—1,n— atk 
k=0 
=14+C(n, Dax+C(n +1, 2)a7x? +--- 
4 x2 x3 


Note: The series for the last two generating functions can be found in most calculus books when power series are discussed. 


EXAMPLE 11 


EXAMPLE 12 
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This follows because we obtain a term equal to x!” in the product by picking a term in the 
first sum x°!, a term in the second sum x, and a term in the third sum x, where the 
exponents e1, €2, and e3 satisfy the equation e; + e2 + e3 = 17 and the given constraints. 

It is not hard to see that the coefficient of x!’ in this product is 3. Hence, there are 
three solutions. (Note that the calculating of this coefficient involves about as much work 
as enumerating all the solutions of the equation with the given constraints. However, the 
method that this illustrates often can be used to solve wide classes of counting problems with 
special formulae, as we will see. Furthermore, a computer algebra system can be used to 
do such computations.) < 


In how many different ways can eight identical cookies be distributed among three distinct 
children if each child receives at least two cookies and no more than four cookies? 


Solution: Because each child receives at least two but no more than four cookies, for each child 
there is a factor equal to 


(x? fg op x‘) 


in the generating function for the sequence {c,,}, where c,, is the number of ways to distribute n 
cookies. Because there are three children, this generating function is 


(x? + x? + x4), 


We need the coefficient of x® in this product. The reason is that the x® terms in the expansion 
correspond to the ways that three terms can be selected, with one from each factor, that have 
exponents adding up to 8. Furthermore, the exponents of the term from the first, second, and 
third factors are the numbers of cookies the first, second, and third children receive, respectively. 
Computation shows that this coefficient equals 6. Hence, there are six ways to distribute the 
cookies so that each child receives at least two, but no more than four, cookies. < 


Use generating functions to determine the number of ways to insert tokens worth $1, $2, 
and $5 into a vending machine to pay for an item that costs r dollars in both the cases when 
the order in which the tokens are inserted does not matter and when the order does matter. (For 
example, there are two ways to pay for an item that costs $3 when the order in which the tokens 
are inserted does not matter: inserting three $1 tokens or one $1 token and a $2 token. When 
the order matters, there are three ways: inserting three $1 tokens, inserting a $1 token and then 
a $2 token, or inserting a $2 token and then a $1 token.) 


Solution: Consider the case when the order in which the tokens are inserted does not matter. 
Here, all we care about is the number of each token used to produce a total of r dollars. Because 
we can use any number of $1 tokens, any number of $2 tokens, and any number of $5 tokens, 
the answer is the coefficient of x” in the generating function 


(ante +e 5) ee oa ee 4 Se oe eg as), 


(The first factor in this product represents the $1 tokens used, the second the $2 tokens used, and 
the third the $5 tokens used.) For example, the number of ways to pay for an item costing $7 
using $1, $2, and $5 tokens is given by the coefficient of x’ in this expansion, which equals 6. 

When the order in which the tokens are inserted matters, the number of ways to insert 
exactly n tokens to produce a total of r dollars is the coefficient of x” in 


ae ea", 
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EXAMPLE 13 


because each of the r tokens may be a $1 token, a $2 token, or a $5 token. Because any number 
of tokens may be inserted, the number of ways to produce r dollars using $1, $2, or $5 tokens, 
when the order in which the tokens are inserted matters, is the coefficient of x” in 


1 
1—(« + x2 4x5) 
1 


1—x —x2— x5’ 


L+(etx? txt (re trx? $Y pe 


where we have added the number of ways to insert 0 tokens, 1 token, 2 tokens, 3 tokens, and 
so on, and where we have used the identity 1/(1 — x) = 1+x+.x?+--- with x replaced 
with x + x? + x°. For example, the number of ways to pay for an item costing $7 using $1, $2, 
and $5 tokens, when the order in which the tokens are used matters, is the coefficient of x’ in this 
expansion, which equals 26. [Hint: To see that this coefficient equals 26 requires the addition 
of the coefficients of x’ in the expansions (x + x? + x°)* for 2 < k < 7. This can be done by 
hand with considerable computation, or a computer algebra system can be used.] < 


Example 13 shows the versatility of generating functions when used to solve problems with 
differing assumptions. 


Use generating functions to find the number of k-combinations of a set with n elements. Assume 
that the binomial theorem has already been established. 


Solution: Each of the n elements in the set contributes the term (1 + x) to the generating function 


f@)= Dd 36 a,x*. Here f(x) is the generating function for {az}, where ag represents the 
number of k-combinations of a set with n elements. Hence, 


f@)=(U+x)". 
But by the binomial theorem, we have 


f(x) = ye (;) rug 


k=0 


n\ n! 
(;) ~ kin — kV 


Hence, C(n, k), the number of k-combinations of a set with n elements, is 


n! 


k(n — ky! < 


Remark: We proved the binomial theorem in Section 6.4 using the formula for the number of 
r-combinations of a set with n elements. This example shows that the binomial theorem, which 
can be proved by mathematical induction, can be used to derive the formula for the number of 
r-combinations of a set with n elements. 


EXAMPLE 14 


EXAMPLE 15 
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Use generating functions to find the number of r-combinations from a set with n elements when 
repetition of elements is allowed. 


Solution: Let G(x) be the generating function for the sequence {a,}, where a, equals the number 
of r-combinations of a set with n elements with repetitions allowed. That is, G(x) = }°7° 9 a-x". 
Because we can select any number of a particular member of the set with n elements 
when we form an r-combination with repetition allowed, each of the n elements contributes 
(ee oe toa product expansion for G(x). Each element contributes this factor 
because it may be selected zero times, one time, two times, three times, and so on, when an 
r-combination is formed (with a total of r elements selected). Because there are n elements in 
the set and each contributes this same factor to G(x), we have 


Go) =(1+x4+x74+-.-)", 
As long as |x| < 1, we have 1 + x +x? +---=1/(1—~), so 
G(x) = 1/d—x)"=(—-x)™. 


Applying the extended binomial theorem (Theorem 2), it follows that 


d= "=(+Cn"=)> (;") (=x)". 


r=0 


The number of r-combinations of a set with n elements with repetitions allowed, when r is a 
positive integer, is the coefficient a, of x” in this sum. Consequently, using Example 8 we find 
that a, equals 


(;") lf =(-1)'Cat+r=L7)>Cly 


=C(n+r-1,r). 4 


Note that the result in Example 14 is the same result we stated as Theorem 2 in Section 6.5. 


Use generating functions to find the number of ways to select r objects of n different kinds if 
we must select at least one object of each kind. 


Solution: Because we need to select at least one object of each kind, each of the n kinds of objects 
contributes the factor (x + x2 + x3 +---) tothe generating function G(x) for the sequence {a;}, 
where a, is the number of ways to select r objects of n different kinds if we need at least one 
object of each kind. Hence, 


G(x) = (tx? taP tee PM axMltxtx? te ax" —x)". 
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Using the extended binomial theorem and Example 8, we have 


G(x) = x"/U — x)" 


=x".(1—x)" 


n = —n r 
("Jeo 
0 


r= 


II 
Se 


We (-1)’C(at+r—1,r)(-1)'x" 
-=0 


r= 


(oe) 

= Yo Cin+r- Lr" 
CO 

= Y\ce-Lt-n)x' 


CO 
= )0Cir-1,r-n)x’. 


r=n 


We have shifted the summation in the next-to-last equality by setting + = n +r sothatt =n 
when r = 0 andn +r — 1 =t — 1, and then we replaced ¢ by r as the index of summation in 
the last equality to return to our original notation. Hence, there are C(r — 1,r — n) ways to 
select r objects of n different kinds if we must select at least one object of each kind. < 


Using Generating Functions to Solve Recurrence Relations 


We can find the solution to a recurrence relation and its initial conditions by finding an explicit 
formula for the associated generating function. This is illustrated in Examples 16 and 17. 


EXAMPLE 16 — Solve the recurrence relation a, = 3ax_, fork = 1, 2,3,... and initial condition ap = 2. 


Solution: Let G(x) be the generating function for the sequence {ax}, thatis, G(x) = yo 0 axe. 
Extra oy First note that 


Examples Qi 
CO [e) 
xG(x) = > agx*t! = ye ap_1x*. 
k=0 k=1 


Using the recurrence relation, we see that 
CO CO 
G(x) — 3xG(x) = ¥ ayx* —3 2 ap_x* 
k=0 k=1 


CO 
=ao+ s (ax — 3ax_1)x* 
k=1 
—2. 


because ag = 2 and ag = 3ax_ 1. Thus, 


G(x) — 3xG(x) = (1 — 3x)G(x) = 2. 


EXAMPLE 17 
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Solving for G(x) shows that G(x) = 2/(1 — 3x). Using the identity 1/(1 — ax) = ae, a®x*, 
from Table 1, we have 


Ce lo) 
Goe2) a> a 
k=0 k=0 
Consequently, ax = 2 - 3. < 


Suppose that a valid codeword is an n-digit number in decimal notation containing an even 
number of Os. Let a, denote the number of valid codewords of length n. In Example 4 of 
Section 8.1 we showed that the sequence {a,,} satisfies the recurrence relation 


An = 8dy—1 + 10°" 


and the initial condition a; = 9. Use generating functions to find an explicit formula for ay. 


Solution: To make our work with generating functions simpler, we extend this sequence 
by setting a9 = 1; when we assign this value to ap and use the recurrence relation, we 
have ay = 8a) + 10° = 8+ 1 = 9, which is consistent with our original initial condition. (It 
also makes sense because there is one code word of length O—the empty string.) 

We multiply both sides of the recurrence relation by x” to obtain 


Anx" = Ban_jx" +107 !x", 


Let G(x) = Yi anx" be the generating function of the sequence ag, a1, a2,.... We sum 
both sides of the last equation starting with n = 1, to find that 


CO [o.@) 
G(x) -1= > ax = S-(8an—1" 4 10" 1x") 


n=1 n=1 


oe) oe) 
= 8) dn—1x" he » 107! x" 
n=1 


n=1 


oo ioe) 
= 8x er ae +x ». 10° !,7-1 
n=1 


n=1 
oo 


CO 
= 8x 2 Anx" +x > 10" x" 
n=0 n=0 
= 8xG(x) + x/C1 — 10x), 
where we have used Example 5 to evaluate the second summation. Therefore, we have 
G(x) — 1 = 8xG(x) + x/(1 — 10x). 


Solving for G(x) shows that 


1 — 9x 


GO) = Ged — 10x)" 
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EXAMPLE 18 


Expanding the right-hand side of this equation into partial fractions (as is done in the integration 
of rational functions studied in calculus) gives 


a 
= oN Tee T= tet 


Using Example 5 twice (once with a = 8 and once with a = 10) gives 


1 CO CO 
G@)=5 ( > Bx ae > ws 
n=0 n=0 


= 1 
= D> =(8" + 10")x”. 
2 
Consequently, we have shown that 


i n n 


Proving Identities via Generating Functions 


In Chapter 6 we saw how combinatorial identities could be established using combinatorial 
proofs. Here we will show that such identities, as well as identities for extended binomial coef- 
ficients, can be proved using generating functions. Sometimes the generating function approach 
is simpler than other approaches, especially when it is simpler to work with the closed form 
of a generating function than with the terms of the sequence themselves. We illustrate how 
generating functions can be used to prove identities with Example 18. 


Use generating functions to show that 
n 
CGH = Cn, n) 
k=0 
whenever n is a positive integer. 


Solution: First note that by the binomial theorem C (2n, n) is the coefficient of x” in (1 + xy", 
However, we also have 


(1+x)"=[0+x)"? 
= [C(n, 0) + C(n, Ix + C(n, 2)x? +++» + CC, n)x"P. 
The coefficient of x” in this expression is 
C(n, 0)C(n,n) + C(n, 1)C(n,n — 1) + C(n, 2)C(n,n — 2) +--+ + C(n,n)C(n, 0). 


This equals }°;_9 C(n, k)*, because C(n,n —k) = C(n,k). Because both C(2n,n) and 
> p=0 CM, k)? represent the coefficient of x” in (1 + x)”, they must be equal. < 


Exercises 42 and 43 ask that Pascal’s identity and Vandermonde’s identity be proved using 
generating functions. 


Exercises 


1. Find the generating function for the finite sequence 2, 2, 
2, 2, 2,2. 

2. Find the generating function for the finite sequence 1, 4, 
16, 64, 256. 

In Exercises 3—8, by a closed form we mean an algebraic ex- 
pression not involving a summation over a range of values or 
the use of ellipses. 

3. Find a closed form for the generating function for each 
of these sequences. (For each sequence, use the most ob- 
vious choice of a sequence that follows the pattern of the 
initial terms listed.) 


a) 0, 2,2, 2,2, 2, 2, 0, 0, 0, 0, 0, 
b) 0,0,0,1,1,1,1,1,1,... 

c) 0,1, 0,0, 1, 0,0, 1,0, 0, 1,. 
d) 2,4, 8, 16, 32, 64, 128, 256,... 

(3): (7): Gh G).20009 
e) oh ap laden La] 0 0 0,00... 
f) 2,-—2,2,—2,2,—2,2,—-2,... 

g) 1,1,0,1,1,1,1,1,1,1, 


h) 0, 0,0, 1, 2,3,4,... 

4. Find a closed form for the generating function for each 
of these sequences. (Assume a general form for the terms 
of the sequence, using the most obvious choice of such a 


sequence.) 

a) —1, —1, —1, —1, —1, —1, —1, 0, 0,0, 0,0,0,... 
b) 1,3, 9, 27, 81, 243, 729, 

c) 0, 0, 3, —3, 3, —3, 3, —3, 

d) 1,2,1,1,1,1,1,1,1,... 

» (7) 0f7),2(7 27(7) .0,0,0,0 
e Gy at? ag peers pena a Mss res 
£) =3,3;53,3,=3,.3 


g) 0,1, —2, 4, —8, 16, —32, 64,... 
h) 1,0, 1,0, 1,0, 1,0,... 
5. Find a closed form for the generating function for the 
sequence {a,}, where 
a) a, = 5 foralln =0,1,2,.... 
b) a, = 3” for alln = 0,1, 2,.... 
C) adn = 2 forn = 3,4,5,...andag = a; = a2 = 0. 
d) a, = 2n+3 foralln =0,1,2,.... 


8 
e) dn = (*) for alln = 0,1,2,.... 


n+4 
f) a, = for alln = 0,1,2,.... 
n 


6. Find a closed form for the generating function for the 
sequence {a,}, where 
a) a, = —1 foralln=0,1,2,.... 
b) a, = 2” forn = 1, 2,3,4,... and ap = 0. 
c) dg =n—1forn=0,1,2,.... 
d) a, = 1/(n+ 1)! forn = 0,1, 2,.... 


ate (5) fot = 0, 1,2,.,.: 


10 
f) = ( 41) BEM =O AZo 
n 


7 


8 


9 


10 


11 


12 


13 


14 


15 
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. For each of these generating functions, provide a closed 
formula for the sequence it determines. 
a) (3x —4)3 b) (x + 1 
ce) 1/( —5x) d) x3/(1+3x) 
e) x7 +3x +74 (1/(1 — x’)) 
f) @4/( —x4))— x3 —x?-x-1 
g) x*/(1—x)? h) 2e”* 


. For each of these generating functions, provide a closed 
formula for the sequence it determines. 
a) (x2 +1)? b) Gx — 1)3 
ce) 1/(1 — 2x?) d) x7/(1— x) 
e) x -14+(/d—3x)) f) (+x3)/d +x) 
#9) x/(1+x +x) h) e* —1 


. Find the coefficient of x!° in the power series of each of 
these functions. 
a) 14+x° 4x10 435 4...53 
b) Qi +xtt xh txOta7 +...) 
ce) (xt $08 4 x3 txt tt x8 t x71 tx4 
x27 + x3 4a44...) 
d) (x7 4x44 x64 x8 4---)3 4 x6 4 x9 4 
(xt +x8 4 x12 4...) 
e) tart xt x94 x84... txt x84 x? + 
L454 412 4 4184...) 
. Find the coefficient of x? in the power series of each of 
these functions. 
a) (l+x34+x°4x°+.---)3 
b) 2 tx3 txt gah +36 4---)3 
ce) (xP +29 4+ x3 + xt(x ta? 4x3 4x4 4---) 
d) (x txt txt 4+ xO -. yx? xt exo 4 x8 4 


e) (+x + x2)3 


. Find the coefficient of x!° in the power series of each of 
these functions. 
a) 1/(1 — 2x) b) 1/ +x)? 
ce) 1/0 —x)3 d) 1/(1+2x)4 


e) x4/(1 — 3x)3 

. Find the coefficient of x!? in the power series of each of 
these functions. 
a) 1/(1+3x) b) 1/(1 — 2x)? 
ce) 1/d+x)8 d) 1/(1— 4x)? 
e) x7/(1+ 4x) 

. Use generating functions to determine the number of dif- 
ferent ways 10 identical balloons can be given to four 
children if each child receives at least two balloons. 


. Use generating functions to determine the number of dif- 
ferent ways 12 identical action figures can be given to five 
children so that each child receives at most three action 
figures. 


. Use generating functions to determine the number of dif- 
ferent ways 15 identical stuffed animals can be given to 
six children so that each child receives at least one but no 
more than three stuffed animals. 
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16. Use generating functions to find the number of ways to 
choose a dozen bagels from three varieties—egg, salty, 
and plain—if at least two bagels of each kind but no more 
than three salty bagels are chosen. 

17. In how many ways can 25 identical donuts be distributed 
to four police officers so that each officer gets at least 
three but no more than seven donuts? 

18. Use generating functions to find the number of ways to 
select 14 balls from a jar containing 100 red balls, 100 
blue balls, and 100 green balls so that no fewer than 3 and 
no more than 10 blue balls are selected. Assume that the 
order in which the balls are drawn does not matter. 

19. What is the generating function for the sequence {cx}, 
where cx is the number of ways to make change for k 
dollars using $1 bills, $2 bills, $5 bills, and $10 bills? 

20. What is the generating function for the sequence {cx}, 
where cz represents the number of ways to make change 
for k pesos using bills worth 10 pesos, 20 pesos, 50 pesos, 
and 100 pesos? 

21. Give a combinatorial interpretation of the coefficient 
of x* in the expansion (1 + x +x?2+x3+---)3. Use 
this interpretation to find this number. 

22. Give a combinatorial interpretation of the coefficient 
of x° in the expansion (1 +x +x? +2x3+.---)". Use 
this interpretation to find this number. 

23. a) What is the generating function for {a,x}, where ax 

is the number of solutions of x; + x2 + x3 = k when 
X1,X2, and x3 are integers with x; > 2,0 < x2 < 3, 
and 2 < x3 < 5? 

b) Use your answer to part (a) to find dg. 

24. a) What is the generating function for {a,}, where ax 
is the number of solutions of x; + x2 +x3+4%4 =k 
when x1, x2, x3, and x4 are integers with x; > 3, 
1 <x <5,0< 1x3 <4,andx4> 1? 

b) Use your answer to part (a) to find a7. 

25. Explain how generating functions can be used to find the 
number of ways in which postage of r cents can be pasted 
on an envelope using 3-cent, 4-cent, and 20-cent stamps. 
a) Assume that the order the stamps are pasted on does 

not matter. 

b) Assume that the stamps are pasted in a row and the 
order in which they are pasted on matters. 

c) Use your answer to part (a) to determine the number 
of ways 46 cents of postage can be pasted on an en- 
velope using 3-cent, 4-cent, and 20-cent stamps when 
the order the stamps are pasted on does not matter. 
(Use of a computer algebra program is advised.) 

d) Use your answer to part (b) to determine the num- 
ber of ways 46 cents of postage can be pasted in a 
row on an envelope using 3-cent, 4-cent, and 20-cent 
stamps when the order in which the stamps are pasted 
on matters. (Use of a computer algebra program is 
advised.) 

26. a) Show that 1/(1 —x —x?—x3 —x4—x° — x®) is 
the generating function for the number of ways that 
the sum n can be obtained when a die is rolled repeat- 
edly and the order of the rolls matters. 


b) Use part (a) to find the number of ways to roll a total 
of 8 when a die is rolled repeatedly, and the order of 
the rolls matters. (Use of a computer algebra package 
is advised.) 

27. Use generating functions (and a computer algebra pack- 
age, if available) to find the number of ways to make 
change for $1 using 
a) dimes and quarters. 

b) nickels, dimes, and quarters. 

c) pennies, dimes, and quarters. 

d) pennies, nickels, dimes, and quarters. 

28. Use generating functions (and a computer algebra pack- 
age, if available) to find the number of ways to make 
change for $1 using pennies, nickels, dimes, and quarters 
with 
a) no more than 10 pennies. 

b) no more than 10 pennies and no more than 10 nickels. 

*c) no more than 10 coins. 

29. Use generating functions to find the number of ways to 
make change for $100 using 
a) $10, $20, and $50 bills. 

b) $5, $10, $20, and $50 bills. 

c) $5, $10, $20, and $50 bills if at least one bill of each 
denomination is used. 

d) $5, $10, and $20 bills if at least one and no more than 
four of each denomination is used. 

30. If G(x) is the generating function for the sequence {ax}, 
what is the generating function for each of these se- 


quences? 
a) 2a, 2a, 2a2, 2a3, Sits 
b) 0, ao, a1, a2, a3, ... (assuming that terms follow the 


pattern of all but the first term) 
c) 0, 0, 0, 0, a2, a3, ... (assuming that terms follow the 
pattern of all but the first four terms) 
d) az, a3, 44,... 
e) at, 2a2, 3a3, 4a4, ... [Hint: Calculus required here.] 
f) a, 2a0a1, at + 2aga2, 2a9a3 + 2a,a2, 2aga4 + 
2a;a3 + as, wave 
31. If G(x) is the generating function for the sequence {ax}, 
what is the generating function for each of these se- 
quences? 
a) 0, 0, 0, a3, a4, a5, .. . (assuming that terms follow the 
pattern of all but the first three terms) 
b) ao, 0, a1, 0, a2, 0,... 
c) 0, 0, 0, 0, ao, a1, a2, ... (assuming that terms follow 
the pattern of all but the first four terms) 
d) ao, 2a), 4a2, 8a3, 16a4,... 
e) 0, dao, a1/2, a2/3, a3/4, ... [Hint: Calculus required 
here. ] 
f) ao,do + 41,49 +4, +42,a9 +a, +42 +43,... 
32. Use generating functions to solve the recurrence relation 
ay = 7ag—, with the initial condition ag = 5. 
33. Use generating functions to solve the recurrence relation 
ag = 3az— 1 + 2 with the initial condition ap = 1. 
34. Use generating functions to solve the recurrence relation 
ay = 3ax—1 + 44! with the initial condition aj = 1. 


35. 


36. 


37. 


38. 


39. 


* 40. 


*41, 


42. 


43. 


44. 


Use generating functions to solve the recurrence rela- 
tion ag = 5ag_ 1 — 6ax—2 with initial conditions ap = 6 
and a; = 30. 

Use generating functions to solve the recurrence relation 
ak = ag, + 2ag_2 + 2* with initial conditions ag = 4 
= 12. 

Use generating functions to solve the recurrence relation 
ap = 4az—) — 4ag_2 + k* with initial conditions ag = 2 
=). 

Use generating functions to solve the recurrence re- 
lation ag = 2ag_1 + 3ax_2 + 4* + 6 with initial condi- 
tions ag = 20, a, = 60. 


anda 


and a 


Use generating functions to find an explicit formula for 
the Fibonacci numbers. 


a) Show that if n is a positive integer, then 


Core 


n ) (-4" 


b) Use the extended binomial theorem and part (a) to 
show that the coefficient of x” in the expansion of 
(1 —4x)7!/? is a) for all nonnegative integers n. 

(Calculus required ) Let {C,,} be the sequence of Catalan 

numbers, that is, the solution to the recurrence relation 

Cn = Dhao CeCn—k-1 with Co = C) = 1 (see Exam- 

ple 5 in Section 8.1). 

a) Show that if G(x) is the generating function for the se- 
quence of Catalan numbers, then xG(x)* — G(x) + 
1 = 0. Conclude (using the initial conditions) that 
G(x) = A -— V1 —4x)/(2x). 


b) Use Exercise 40 to conclude that 


i Ate” #2 
Ce De aa (ae 


n=0 


so that 


C= 1 2n 
"nt 1\an SJ? 


c) Show that C, > 2”~! for all positive integers n. 

Use generating functions to prove Pascal’s identity: 

C(n,r) = C(n—1,r) + C(n— 1,r — 1) whenn andr 

are positive integers with r < n. [Hint: Use the identity 

(+x)? =(1+x)"!4+x(4+x)""!] 

Use generating functions to prove Vandermonde’s iden- 

tity: C(m+n,r) = Yop) C(m,r — k)C(n, k), when- 

ever m,n, and r are nonnegative integers with r not ex- 

ceeding either m or n. [Hint: Look at the coefficient of x” 

in both sides of (1 + x)”™" = (1+ x)™(1+x)".] 

This exercise shows how to use generating functions to 

derive a formula for the sum of the first n squares. 

a) Show that («2+ x)/(1—x)* is the 
ating function for the sequence {ay}, 
Gy = 12 +227 4.---47n?. 

b) Use part (a) to find an explicit formula for the sum 
127 +274.---4n?. 


gener- 
where 
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The exponential generating function for the sequence {a,} 
is the series 


For example, the exponential generating function for the 


sequence 1, 1, 1,... 


is the function )°°0 9 x"/n! =e". 


(You will find this particular series useful in these exercises.) 
Note that e* is the (ordinary) generating function for the se- 
quence 1, 1, 1/2!, 1/3!, 1/4!,.... 


45. 


46. 


47. 


48. 


49. 


*50. 


Find a closed form for the exponential generating func- 
tion for the sequence {a,}, where 

b) a, = (-1)". 

d) gq =n+l. 


a) a, = 2. 

c) ad, = 3". 

e) ad, =1/M+1). 

Find a closed form for the exponential generating func- 

tion for the sequence {a,,}, where 

a) dy, = (—2)". b) a, = —1. 

C) dy =n. d) a, =n(n — 1). 

€) ay) = 1/(a+ I)(n + 2)). 

Find the sequence with each of these functions as its ex- 

ponential generating function. 

a) f(x) =e * b) f(x) = 3x”* 

c) f(x)=e*—3e* = d) f(x) =(1-x) +e 

e) f@)=e * —C/d—2)) 

f) f(x) =e7* — (14x) + (1/( — 2x) 

9 faa 

Find the sequence with each of these functions as its ex- 

ponential generating function. 

a) f(x) =e* b) f(x) = 2e734! 

ec) f@=e™t+e* — d) f(@*) =(1+2x) +e* 

e) fa) =e*—- (1/1 +x)) : 

f) f(x) =xe* gs) f@=e 

A coding system encodes messages using strings of octal 

(base 8) digits. A codeword is considered valid if and only 

if it contains an even number of 7s. 

a) Finda linear nonhomogeneous recurrence relation for 
the number of valid codewords of length n. What are 
the initial conditions? 

b) Solve this recurrence relation using Theorem 6 in Sec- 
tion 8.2. 

c) Solve this recurrence relation using generating func- 
tions. 


A coding system encodes messages using strings of 
base 4 digits (that is, digits from the set {0, 1, 2, 3}). 
A codeword is valid if and only if it contains an even 
number of Os and an even number of 1s. Let a, equal 
the number of valid codewords of length n. Furthermore, 
let by, Cn, and d,, equal the number of strings of base 4 dig- 
its of length n with an even number of Os and an odd num- 
ber of 1s, with an odd number of Os and an even number 
of 1s, and with an odd number of Os and an odd number 
of 1s, respectively. 
a) Show that d, = 4” — ayn — by — cn. Use this to show 
that = dy41 = 2d, + bn + Cn, Dag = Dn — Ch + 4", 
and Cy41 = Cn — bn +4". 
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b) What are a1, D1, cy, and d,? 
c) Use parts (a) and (b) to find a3, b3, c3, and d3. 


d) Use the recurrence relations in part (a), together with 
the initial conditions in part (b), to set up three equa- 
tions relating the generating functions A(x), B(x), 
and C(x) for the sequences {a,}, {b,}, and {c,}, re- 
spectively. 

e) Solve the system of equations from part (d) to get 
explicit formulae for A(x), B(x), and C(x) and use 
these to get explicit formulae for dy, by, Cn, and dy. 


Generating functions are useful in studying the number of 
different types of partitions of an integer n. A partition of 
a positive integer is a way to write this integer as the sum 
of positive integers where repetition is allowed and the order 
of the integers in the sum does not matter. For example, the 
partitions of 5 (with no restrictions) are 1+1+1+1+1, 
1+14+1+2, 1+1+4+3, 14242, 144, 2+3, and 5. 
Exercises 51-56 illustrate some of these uses. 


51. 


52. 


53. 


54, 


55. 


Show that the coefficient p(n) of x” in the formal 
power series expansion of 1/((1—x)(1 —x?)(1—x3)---) 
equals the number of partitions of n. 


Show that the coefficient p,(n) of x” in the formal 
power series expansion of 1/((1—x)(1 —x)\(1 —x) see) 
equals the number of partitions of n into odd integers, that 
is, the number of ways to write n as the sum of odd positive 
integers, where the order does not matter and repetitions 
are allowed. 


Show that the coefficient pg(n) of x” in the formal power 
series expansion of (1+ x)(1 +x2)(1+ x3) +++ equals 
the number of partitions of n into distinct parts, that is, 
the number of ways to write n as the sum of positive in- 
tegers, where the order does not matter but no repetitions 
are allowed. 


Find p,(n), the number of partitions of n into odd parts 
with repetitions allowed, and pg(n), the number of par- 
titions of n into distinct parts, for 1 < n < 8, by writing 
each partition of each type for each integer. 


Show that if n is a positive integer, then the number of 
partitions of n into distinct parts equals the number 
of partitions of n into odd parts with repetitions allowed; 


Inclusion—Exclusion 


** 56, 


that is, po(n) = pa(n). [Hint: Show that the generating 
functions for p(n) and pg(n) are equal.] 

(Requires calculus) Use the generating function of p(n) 
to show that p(n) < e©V" for some constant C. [Hardy 
and Ramanujan showed that p(n) ~ e” V273/n /(4V3n), 
which means that the ratio of p(n) and the right-hand side 
approaches | as n approaches infinity. ] 


Suppose that X is a random variable on a sample space S such 
that X(s) is a nonnegative integer for all s €¢ S. The proba- 


bility generating function for X is 


57. 


58. 


59. 


60. 


Gx(x) = D7 pCX(s) = Kx". 


k=0 


(Requires calculus) Show that if Gx is the probability 
generating function for a random variable X such that 
X(s) is a nonnegative integer for all s € S, then 

a) Gy(I)=1. b) E(X) = GY (J). 

ce) V(X) = G1) + G1) — G1). 

Let X be the random variable whose value is n if the 

first success occurs on the nth trial when independent 

Bernoulli trials are performed, each with probability of 

success p. 

a) Find a closed formula for the probability generating 
function Gy. 

b) Find the expected value and the variance of X using 
Exercise 57 and the closed form for the probability 
generating function found in part (a). 

Let m be a positive integer. Let X,, be the random vari- 

able whose value is n if the mth success occurs on the 

(n + m)th trial when independent Bernoulli trials are per- 

formed, each with probability of success p. 

a) Using Exercise 32 in the Supplementary Exercises 
of Chapter 7, show that the probability generating 
function Gy,, is given by Gy,, (x) = p”/(1 — qx)”, 
where g = 1 — p. 

b) Find the expected value and the variance of X, using 
Exercise 57 and the closed form for the probability 
generating function in part (a). 

Show that if X and Y are independent random variables on 

asample space S such that X (s) and Y (s) are nonnegative 

integers for all s € S, then Gy+y(x) = Gx(x)Gy(x). 


Introduction 


A discrete mathematics class contains 30 women and 50 sophomores. How many students 
in the class are either women or sophomores? This question cannot be answered unless more 
information is provided. Adding the number of women in the class and the number of sophomores 
probably does not give the correct answer, because women sophomores are counted twice. This 
observation shows that the number of students in the class that are either sophomores or women is 
the sum of the number of women and the number of sophomores in the class minus the number 
of women sophomores. A technique for solving such counting problems was introduced in 


EXAMPLE 1 


EXAMPLE 2 
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Section 6.1. In this section we will generalize the ideas introduced in that section to solve 
problems that require us to count the number of elements in the union of more than two sets. 


The Principle of Inclusion—Exclusion 


How many elements are in the union of two finite sets? In Section 2.2 we showed that the number 
of elements in the union of the two sets A and B is the sum of the numbers of elements in the 
sets minus the number of elements in their intersection. That is, 


|AU B| = |A| + |B] —|AN BI. 


As we showed in Section 6.1, the formula for the number of elements in the union of two sets 
is useful in counting problems. Examples 1-3 provide additional illustrations of the usefulness 
of this formula. 


In a discrete mathematics class every student is a major in computer science or mathematics, 
or both. The number of students having computer science as a major (possibly along with 
mathematics) 1s 25; the number of students having mathematics as a major (possibly along with 
computer science) is 13; and the number of students majoring in both computer science and 
mathematics is 8. How many students are in this class? 


Solution: Let A be the set of students in the class majoring in computer science and B be the set 
of students in the class majoring in mathematics. Then A / B is the set of students in the class 
who are joint mathematics and computer science majors. Because every student in the class 
is majoring in either computer science or mathematics (or both), it follows that the number of 
students in the class is |A U B|. Therefore, 


|AU B| = |A| + |B] —|AN BI 
= 25+ 13-8 =30. 


Therefore, there are 30 students in the class. This computation is illustrated in Figure 1. < 


How many positive integers not exceeding 1000 are divisible by 7 or 11? 


Solution: Let A be the set of positive integers not exceeding 1000 that are divisible by 7, and 
let B be the set of positive integers not exceeding 1000 that are divisible by 11. Then A U B 
is the set of integers not exceeding 1000 that are divisible by either 7 or 11, and A B is the 
set of integers not exceeding 1000 that are divisible by both 7 and 11. From Example 2 of 
Section 4.1, we know that among the positive integers not exceeding 1000 there are | 1000/7] 
integers divisible by 7 and [1000/11] divisible by 11. Because 7 and 11 are relatively prime, 
the integers divisible by both 7 and 11 are those divisible by 7- 11. Consequently, there are 
[1000/(11-7)] positive integers not exceeding 1000 that are divisible by both 7 and 11. It 
follows that there are 


|AU B| = |A|+ |B) —|ANB 
_ | 1000 : 1000 | _ | 1000 
| 7 11 7-11 
= 142+ 90 — 12 = 220 


positive integers not exceeding 1000 that are divisible by either 7 or 11. This computation is 
illustrated in Figure 2. < 
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|A U B|=|A]+|B|-|A N Bl=25 + 13 — 8 =30 IA U Bl = IAI+IBI-IA N Bl =142 + 90-12 = 220 


|A|=25 |AM Bl=8 |B\= 13 |Al = 142 IAM Bl =12 |B| =90 
FIGURE 1 The Set of Students in a FIGURE 2 The Set of Positive Integers Not 
Discrete Mathematics Class. Exceeding 1000 Divisible by Either 7 or 11. 
Example 3 shows how to find the number of elements in a finite universal set that are outside 
the union of two sets. 
EXAMPLE3 Suppose that there are 1807 freshmen at your school. Of these, 453 are taking a course in 


computer science, 567 are taking a course in mathematics, and 299 are taking courses in both 
computer science and mathematics. How many are not taking a course either in computer science 
or in mathematics? 


Solution: To find the number of freshmen who are not taking a course in either mathematics 
or computer science, subtract the number that are taking a course in either of these subjects 
from the total number of freshmen. Let A be the set of all freshmen taking a course in com- 
puter science, and let B be the set of all freshmen taking a course in mathematics. It follows 
that |A| = 453, |B| = 567, and |AM B| = 299. The number of freshmen taking a course in 
either computer science or mathematics is 


|AU B| = |A| + |B] —|AN B| = 453 + 567 — 299 = 721. 


Consequently, there are 1807 — 721 = 1086 freshmen who are not taking a course in computer 
science or mathematics. < 


We will now begin our development of a formula for the number of elements in the union 
of a finite number of sets. The formula we will develop is called the principle of inclusion- 
exclusion. For concreteness, before we consider unions of n sets, where n is any positive integer, 
we will derive a formula for the number of elements in the union of three sets A, B, and C. To 
construct this formula, we note that |A| + |B] + |C| counts each element that is in exactly one 
of the three sets once, elements that are in exactly two of the sets twice, and elements in all three 
sets three times. This is illustrated in the first panel in Figure 3. 

To remove the overcount of elements in more than one of the sets, we subtract the number 
of elements in the intersections of all pairs of the three sets. We obtain 


|A| + |B] +/C]—|AN B|-|ANC|—|BNCl. 


This expression still counts elements that occur in exactly one of the sets once. An element that 
occurs in exactly two of the sets is also counted exactly once, because this element will occur 
in one of the three intersections of sets taken two at a time. However, those elements that occur 
in all three sets will be counted zero times by this expression, because they occur in all three 
intersections of sets taken two at a time. This is illustrated in the second panel in Figure 3. 
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(a) Count of elements by (b) Count of elements by (c) Count of elements by 
|A|+|B|+|C |A|+|B|+|C|-|AN Bl- |A|+|B|+|C|-|AN Bl- 
|ANC|-|BNC| |ANc|-|BNc|+|ANBNC| 


FIGURE 3 Finding a Formula for the Number of Elements in the Union of Three Sets. 


EXAMPLE 4 


To remedy this undercount, we add the number of elements in the intersection of all three 
sets. This final expression counts each element once, whether it is in one, two, or three of the 
sets. Thus, 


|AUBUC|=|A]+|B|+|C]|-|ANB|—|ANC|—-|BNC|+|ANBNC. 


This formula is illustrated in the third panel of Figure 3. 
Example 4 illustrates how this formula can be used. 


A total of 1232 students have taken a course in Spanish, 879 have taken a course in French, 
and 114 have taken a course in Russian. Further, 103 have taken courses in both Spanish and 
French, 23 have taken courses in both Spanish and Russian, and 14 have taken courses in both 
French and Russian. If 2092 students have taken at least one of Spanish, French, and Russian, 
how many students have taken a course in all three languages? 

Solution: Let S be the set of students who have taken a course in Spanish, F the set of students 
who have taken a course in French, and R the set of students who have taken a course in Russian. 
Then 


|S] = 1232, |F|=879, |R|=114, 
ISA F| = 103, |S R| = 23, |FN R| = 14, 


and 
|SU FU R| = 2092. 
When we insert these quantities into the equation 
ISUFUR|=|S|+|F[+|R|-|SOF|—-|SOR|-|FOR|[+|SOFOR| 
we obtain 
2092 = 1232 + 879 + 114 — 103 — 23 — 144+ |SN FN RI. 


We now solve for |S F 1 R|. We find that |S  F N R| = 7. Therefore, there are seven students 
who have taken courses in Spanish, French, and Russian. This is illustrated in Figure 4. < 
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THEOREM 1 


ISA FOR|=? ISO F|=103 
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IS R|=23 IFnR|=14 


IR|=114 


S UF UR|=2092 
FIGURE 4 _ The Set of Students Who Have Taken Courses 


in Spanish, French, and Russian. 


We will now state and prove the inclusion—exclusion principle, which tells us how many 
elements are in the union of a finite number of finite sets. 


THE PRINCIPLE OF INCLUSION-EXCLUSION Let A), Ao, ..., A, be finite sets. 
Then 


JA1UAQU---UAnl= D7 |Ai1— DY 142.94]! 
l<i<n l<i<j<n 
+ Do IAIN AN Agl = + CDN ALN A219 An 


1<i<j<k<n 


Proof: We will prove the formula by showing that an element in the union is counted exactly 
once by the right-hand side of the equation. Suppose that a is a member of exactly r of the 
sets Aj, Ao,..., A, where 1 <r <n. This element is counted C(r, 1) times by ©|A;]|. It is 
counted C(r, 2) times by &|A; 1 A;|. In general, it is counted C(r, m) times by the summation 
involving m of the sets A;. Thus, this element is counted exactly 


CG.) =C 2) $63) = + E17 CR) 


times by the expression on the right-hand side of this equation. Our goal is to evaluate this 
quantity. By Corollary 2 of Section 6.4, we have 


C(r, 0) — C(r, 1) + C(r, 2) — ++» + (-1)'C(r, r) = 0. 
Hence, 


1 =C(r,0) = C(r, 1) — C(r,2) +--+» +(-1)"*'C0, Pr). 
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Therefore, each element in the union is counted exactly once by the expression on the right-hand 
side of the equation. This proves the principle of inclusion—exclusion. 4 


The inclusion—exclusion principle gives a formula for the number of elements in the union 
of n sets for every positive integer n. There are terms in this formula for the number of ele- 
ments in the intersection of every nonempty subset of the collection of the n sets. Hence, there 
are 2” — | terms in this formula. 


EXAMPLE 5 Give a formula for the number of elements in the union of four sets. 


Extra ~ Solution: The inclusion—exclusion principle shows that 


Examples 


|Aj U Az U Az U Ag| = |A1| + |A2| + |A3] + [Aa] 
— |A1M A2| — |A1 1 A3| — |A1 9 Ag] — |A2 9 A3| — |A2O Aa] 
— |A3M Ag| + [A171 A2M A3| + [A191 A2N Ag] + |A1 9 A329 Aa 
+ (Ae ti Asti Aal = (Ag A AatiAg Aa, 


Note that this formula contains 15 different terms, one for each nonempty subset of 


{A}, Ao, A3, Ag}. 


Exercises 


. How many elements are in A; U A? if there are 12 ele- 
ments in A,, 18 elements in A2, and 

a) AyNA2 =? b) |A, 9 A2| = 1? 

c) |AyN A2| = 6? d) Aj C Ay? 

. There are 345 students at a college who have taken a 
course in calculus, 212 who have taken a course in dis- 
crete mathematics, and 188 who have taken courses in 
both calculus and discrete mathematics. How many stu- 
dents have taken a course in either calculus or discrete 
mathematics? 


. A survey of households in the United States reveals that 
96% have at least one television set, 98% have telephone 
service, and 95% have telephone service and at least 
one television set. What percentage of households in the 
United States have neither telephone service nor a televi- 
sion set? 


. A marketing report concerning personal computers states 
that 650,000 owners will buy a printer for their machines 
next year and 1,250,000 will buy at least one software 
package. If the report states that 1,450,000 owners will 
buy either a printer or at least one software package, how 
many will buy both a printer and at least one software 
package? 

. Find the number of elements in A; U Az U A3 if there 
are 100 elements in each set and if 

a) the sets are pairwise disjoint. 


b) there are 50 common elements in each pair of sets and 
no elements in all three sets. 


c) there are 50 common elements in each pair of sets and 


25 elements in all three sets. 
d) the sets are equal. 


. Find the number of elements in A, U Az U A3 if there are 


100 elements in Aj, 1000 in Ao, and 10,000 in A3 if 


a) A, C Ao and Az C A3. 

b) the sets are pairwise disjoint. 

c) there are two elements common to each pair of sets 
and one element in all three sets. 


. There are 2504 computer science students at a school. Of 


these, 1876 have taken a course in Java, 999 have taken a 
course in Linux, and 345 have taken a course in C. Fur- 
ther, 876 have taken courses in both Java and Linux, 231 
have taken courses in both Linux and C, and 290 have 
taken courses in both Java and C. If 189 of these students 
have taken courses in Linux, Java, and C, how many of 
these 2504 students have not taken a course in any of 
these three programming languages? 


. Inasurvey of 270 college students, it is found that 64 like 


brussels sprouts, 94 like broccoli, 58 like cauliflower, 26 
like both brussels sprouts and broccoli, 28 like both brus- 
sels sprouts and cauliflower, 22 like both broccoli and 
cauliflower, and 14 like all three vegetables. How many 
of the 270 students do not like any of these vegetables? 


. How many students are enrolled in a course either in cal- 


culus, discrete mathematics, data structures, or program- 
ming languages at a school if there are 507, 292, 312, 
and 344 students in these courses, respectively; 14 in both 
calculus and data structures; 213 in both calculus and pro- 
gramming languages; 211 in both discrete mathematics 
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10. 


11. 


12. 


13. 


*14, 


15. 


16. 


17. 


18. 


19. 


and data structures; 43 in both discrete mathematics and 
programming languages; and no student may take cal- 
culus and discrete mathematics, or data structures and 
programming languages, concurrently? 

Find the number of positive integers not exceeding 100 
that are not divisible by 5 or by 7. 

Find the number of positive integers not exceeding 100 
that are either odd or the square of an integer. 

Find the number of positive integers not exceeding 1000 
that are either the square or the cube of an integer. 

How many bit strings of length eight do not contain six 
consecutive Os? 

How many permutations of the 26 letters of the English 
alphabet do not contain any of the strings fish, rat or bird? 
How many permutations of the 10 digits either begin with 
the 3 digits 987, contain the digits 45 in the fifth and sixth 
positions, or end with the 3 digits 123? 

How many elements are in the union of four sets if 
each of the sets has 100 elements, each pair of the sets 
shares 50 elements, each three of the sets share 25 ele- 
ments, and there are 5 elements in all four sets? 

How many elements are in the union of four sets if the 
sets have 50, 60, 70, and 80 elements, respectively, each 
pair of the sets has 5 elements in common, each triple of 
the sets has 1 common element, and no element is in all 
four sets? 

How many terms are there in the formula for the number 
of elements in the union of 10 sets given by the principle 
of inclusion—exclusion? 

Write out the explicit formula given by the principle of 
inclusion—exclusion for the number of elements in the 
union of five sets. 


20. 


21. 


* 22. 


23. 


24, 


25. 


26. 


27. 


28. 


29. 


How many elements are in the union of five sets if the 
sets contain 10,000 elements each, each pair of sets has 
1000 common elements, each triple of sets has 100 com- 
mon elements, every four of the sets have 10 common 
elements, and there is | element in all five sets? 


Write out the explicit formula given by the principle of 
inclusion—exclusion for the number of elements in the 
union of six sets when it is known that no three of these 
sets have a common intersection. 


Prove the principle of inclusion—exclusion using mathe- 
matical induction. 


Let E,, Ez, and £3 be three events from a sample space S. 
Find a formula for the probability of E,; U E2 U F3. 


Find the probability that when a fair coin is flipped five 
times tails comes up exactly three times, the first and last 
flips come up tails, or the second and fourth flips come 
up heads. 


Find the probability that when four numbers from 1 to 
100, inclusive, are picked at random with no repetitions 
allowed, either all are odd, all are divisible by 3, or all are 
divisible by 5. 

Find a formula for the probability of the union of four 
events in a sample space if no three of them can occur at 
the same time. 


Find a formula for the probability of the union of five 
events in a sample space if no four of them can occur at 
the same time. 

Find a formula for the probability of the union of n events 
in a sample space when no two of these events can occur 
at the same time. 

Find a formula for the probability of the union of n events 
in a sample space. 


8.6) Applications of Inclusion—Exclusion 


Introduction 


Many counting problems can be solved using the principle of inclusion—exclusion. For instance, 
we can use this principle to find the number of primes less than a positive integer. Many problems 
can be solved by counting the number of onto functions from one finite set to another. The 
inclusion—exclusion principle can be used to find the number of such functions. The famous 
hatcheck problem can be solved using the principle of inclusion—exclusion. This problem asks 
for the probability that no person is given the correct hat back by a hatcheck person who gives 


the hats back randomly. 


An Alternative Form of Inclusion—Exclusion 


There is an alternative form of the principle of inclusion—exclusion that is useful in counting 
problems. In particular, this form can be used to solve problems that ask for the number of 


elements in a set that have none of n properties P;, Po,... 


Pn: 


Let A; be the subset containing the elements that have property P;. The number 


of elements with all the properties P;,, P;,,..., 


P;, will be denoted by N(P;, Pi, ... Pi,)- 


EXAMPLE 1 
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Writing these quantities in terms of sets, we have 
|Ai, NA, N---O Ai,| = NCP;, Pi, .-. Piz). 


If the number of elements with none of the properties P , Po,..., P, is denoted by 
N(P; Ps ... P’) and the number of elements in the set is denoted by N, it follows that 


N(P{P3...P,) =N—|A,UA2U-+-UAgl|. 


From the inclusion—exclusion principle, we see that 


N(P{P)...P))=N—- SY N(P)+ D> NUP) 
l<i<n l<i<j<n 
— 2 NCP; Pj Pe) +--+ + (-D"N(PIP2... Pa). 


1<i<j<k<n 


Example 1 shows how the principle of inclusion—exclusion can be used to determine the 
number of solutions in integers of an equation with constraints. 


How many solutions does 
xX) t2x2 +43 = 11 


have, where x,, x2, and x3 are nonnegative integers with x; < 3, x2 < 4, and x3 < 6? 


Solution: To apply the principle of inclusion—exclusion, let a solution have property P 
if xj > 3, property P) if x2 > 4, and property P3 if x3 > 6. The number of solutions satis- 
fying the inequalities x; < 3, x2 < 4, and x3 < 6is 


N(P{P{P{) = N — N(P1)— N(P2) — N(P3) + N(P1 P23) 
+ N(P}P3) + N(P2P3) — N(P1 P2P3). 


Using the same techniques as in Example 5 of Section 6.5, it follows that 


a N = total number of solutions = C(3 + 11 — 1, 11) = 78, 

N(P,) = (number of solutions with x; > 4) = C(3+7—- 1,7) = CQ, 7) = 36, 

N(P2) = (number of solutions with x2 > 5) = C(3+ 6 — 1, 6) = C(8, 6) = 28, 

N(P3) = (number of solutions with x3 > 7) = C(3+4- 1,4) = C(6,4) = 15, 

N(P, P2) = (number of solutions with x; > 4 and x. > 5) =C(34+2-1,2)= 
C(4, 2) = 6, 

= N(P, P3) = (number of solutions with x; > 4 and x3 > 7) = C(33+0-—1,0) = 1, 

= N(P2P3) = (number of solutions with x2 > 5 and x3 > 7) = 0, 

@ N(P, P2P3) = (number of solutions with x, > 4, x2 > 5, and x3 > 7) = 0. 


Inserting these quantities into the formula for N(P{ PP) shows that the number of solutions 
with x; < 3, x2 <4, and x3 < 6 equals 


N(P{P3P)) = 78 — 36 — 28 -15+6+14+0-0=6. 4 
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EXAMPLE 2 


The Sieve of Eratosthenes 


In Section 4.3 we showed how to use the sieve of Eratosthenes to find all primes less than a 
specified positive integer n. Using the principle of inclusion—exclusion, we can find the number 
of primes not exceeding a specified positive integer with the same reasoning as is used in the 
sieve of Eratosthenes. Recall that a composite integer is divisible by a prime not exceeding 
its square root. So, to find the number of primes not exceeding 100, first note that compos- 
ite integers not exceeding 100 must have a prime factor not exceeding 10. Because the only 
primes not exceeding 10 are 2, 3, 5, and 7, the primes not exceeding 100 are these four primes 
and those positive integers greater than | and not exceeding 100 that are divisible by none 
of 2, 3, 5, or 7. To apply the principle of inclusion—exclusion, let P; be the property that an 
integer is divisible by 2, let Pz be the property that an integer is divisible by 3, let P3 be the 
property that an integer is divisible by 5, and let P4 be the property that an integer is divisible 
by 7. Thus, the number of primes not exceeding 100 is 


4+ N(P; P3P3P,). 


Because there are 99 positive integers greater than | and not exceeding 100, the principle of 
inclusion—exclusion shows that 


N(P; P,P; P,) = 99 — N(P\) — N(P2) — N(P3) — N(Pa) 
+ N(P, Pz) + N(P1 P3) + N(P1 Pa) + N(P2P3) + N(P2P4) + N(P3P4) 
— N(P, P2 P3) — N(P1 P2P4) — N(P1P3P4) — N(P2P3 Pq) 
+ N(P, Po P3 P4). 
The number of integers not exceeding 100 (and greater than 1) that are divisible by all the primes 


in a subset of {2, 3,5, 7} is |100/N]|, where N is the product of the primes in this subset. (This 
follows because any two of these primes have no common factor.) Consequently, 


100 100 100 100 
P! PSPS P!) = 99 — - - 7 
wernt =99-|>]-[r|-[5|- [7] 
| 100], | 100 |, | 100 |, | 100 | | 100 | | | 100 
243 255 rae 578 5.7 5.7 
100 100 100 100_ |, |__ 100 
22355) |20ae7|  |2-as7\ |SeSe0)  |BeSa5e9 


= 99 — 50 — 33-20 —- 14+ 16+10+7+6+4+2—3—2-—1-—-0+0 
= 21. 


Hence, there are 4 + 21 = 25 primes not exceeding 100. 


The Number of Onto Functions 


The principle of inclusion—exclusion can also be used to determine the number of onto functions 
from a set with m elements to a set with n elements. First consider Example 2. 


How many onto functions are there from a set with six elements to a set with three elements? 


Solution: Suppose that the elements in the codomain are b1, b2, and b3. Let P}, Po, and P3 be 
the properties that b;, b2, and b3 are not in the range of the function, respectively. Note that 


THEOREM 1 


Counting onto functions 
is much harder than 
counting one-to-one 
functions! 


EXAMPLE 3 
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a function is onto if and only if it has none of the properties P;, P2, or P3. By the inclusion— 
exclusion principle it follows that the number of onto functions from a set with six elements to 
a set with three elements is 


N(P{P3Ps) = N —[N(P1) + N(P2) + N(P3)] 
+ [N(P} P2) + N(P, P3) + N(P2P3)] — N(P P2P3), 


where N is the total number of functions from a set with six elements to one with three elements. 
We will evaluate each of the terms on the right-hand side of this equation. 

From Example 6 of Section 6.1, it follows that N = 3°. Note that N(P;) is the number of 
functions that do not have b; in their range. Hence, there are two choices for the value of the 
function at each element of the domain. Therefore, N(P;) = 2°. Furthermore, there are C(3, 1) 
terms of this kind. Note that N(P; P;) is the number of functions that do not have b; and 5; 
in their range. Hence, there is only one choice for the value of the function at each element of 
the domain. Therefore, N(P; P; ea = 1° = |. Furthermore, there are C(3, 2) terms of this kind. 
Also, note that N(P; P2 P3) = 0, because this term is the number of functions that have none 
of bj, bz, and b3 in their range. Clearly, there are no such functions. Therefore, the number of 
onto functions from a set with six elements to one with three elements is 


3° — C(3, 1)2° + C(3, 2)1° = 729 — 192 + 3 = 540. 4 


The general result that tells us how many onto functions there are from a set with m elements 
to one with n elements will now be stated. The proof of this result is left as an exercise for the 
reader. 


Let m and n be positive integers with m > n. Then, there are 
eo ee ye a 


onto functions from a set with m elements to a set with n elements. 


An onto function from a set with m elements to a set with n elements corresponds to a 
way to distribute the m elements in the domain to n indistinguishable boxes so that no box is 
empty, and then to associate each of the n elements of the codomain to a box. This means that 
the number of onto functions from a set with m elements to a set with n elements is the number 
of ways to distribute m distinguishable objects to n indistinguishable boxes so that no box is 
empty multiplied by the number of permutations of a set with n elements. Consequently, the 
number of onto functions from a set with m elements to a set with n elements equals n!S(m, n), 
where S(m, 1) 1s a Stirling number of the second kind defined in Section 6.5. This means that 
we can use Theorem | to deduce the formula given in Section 6.5 for S(m, n). (See Chapter 6 
of [MiRo91] for more details about Stirling numbers of the second kind.) 

One of the many different applications of Theorem | will now be described. 


How many ways are there to assign five different jobs to four different employees if every 
employee is assigned at least one job? 


Solution: Consider the assignment of jobs as a function from the set of five jobs to the set of 
four employees. An assignment where every employee gets at least one job is the same as an 
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EXAMPLE 4 


Links 


EXAMPLE 5 


THEOREM 2 


onto function from the set of jobs to the set of employees. Hence, by Theorem | it follows that 
there are 


45 — C(4, 1)3° + C(4, 2)2° — C(4, 3)1° = 1024 — 972 + 192 — 4 = 240 


ways to assign the jobs so that each employee is assigned at least one job. < 


Derangements 


The principle of inclusion—exclusion will be used to count the permutations of n objects that 
leave no objects in their original positions. Consider Example 4. 


The Hatcheck Problem A new employee checks the hats of n people at a restaurant, forgetting 
to put claim check numbers on the hats. When customers return for their hats, the checker gives 
them back hats chosen at random from the remaining hats. What is the probability that no one 
receives the correct hat? 


Remark: The answer is the number of ways the hats can be arranged so that there is no hat in 
its original position divided by n!, the number of permutations of n hats. We will return to this 
example after we find the number of permutations of n objects that leave no objects in their 
original position. 


A derangement is a permutation of objects that leaves no object in its original position. To 
solve the problem posed in Example 4 we will need to determine the number of derangements 
of a set of n objects. 


The permutation 21453 is a derangement of 12345 because no number is left in its original 
position. However, 21543 is not a derangement of 12345, because this permutation leaves 4 
fixed. < 

Let D,, denote the number of derangements of n objects. For instance, D3 = 2, because the 


derangements of 123 are 231 and 312. We will evaluate D,, for all positive integers n, using the 
principle of inclusion—exclusion. 


The number of derangements of a set with n elements is 


Proof: Let a permutation have property P; if it fixes element i. The number of derangements 
is the number of permutations having none of the properties P; fori = 1, 2,...,n. This means 
that 

Dy = NP Po cc Ps) 


Using the principle of inclusion—exclusion, it follows that 


Dn =N—) NCP) +> NPB) — DS NPP PA) +++ + (-D"N(PLP2... Pr), 


i<j i<j<k 


Links 
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where N is the number of permutations of n elements. This equation states that the number of 
permutations that fix no elements equals the total number of permutations, less the number that 
fix at least one element, plus the number that fix at least two elements, less the number that fix 
at least three elements, and so on. All the quantities that occur on the right-hand side of this 
equation will now be found. 


First, note that N = n!, because N is simply the total number of permutations of n elements. 
Also, N(P;) = (n — 1)!. This follows from the product rule, because N(P;) is the number of 
permutations that fix element 7, so the ith position of the permutation is determined, but each 
of the remaining positions can be filled arbitrarily. Similarly, 


N(P Pj) = (n—2)!, 


because this is the number of permutations that fix elements i and j, but where the 
other n — 2 elements can be arranged arbitrarily. In general, note that 


N(P;, Pi, .-. Pi,) = (n—m)!, 


because this is the number of permutations that fix elements i), i2,...,im, but where the 
other n — m elements can be arranged arbitrarily. Because there are C(n, m) ways to choose m 
elements from n, it follows that 


Y> N(P) = CO, D@— I}, 


l<i<n 


Yo NPP) = C(n, 2)(n = 2)!, 


1l<i<j<n 


and in general, 


> N(P;, Pj, «.- Pin) = C(n, m)(n — m)!. 


1<i) <i2<-++<iy<n 
Consequently, inserting these quantities into our formula for D, gives 
D, =n! — C(v, 1I)(n — DY!+ CH, 2)(m — 2)! — +--+ (- 1)" Cm, n)(n — nn)! 


taste n! (n— D1 n! (2-2) EI" n! 0! 
=i ita | ie nol” 


Simplifying this expression gives 


11 wl 


HISTORICAL NOTE In rencontres (matches), an old French card game, the 52 cards in a deck are laid out 
in a row. The cards of a second deck are laid out with one card of the second deck on top of each card of 
the first deck. The score is determined by counting the number of matching cards in the two decks. In 1708 
Pierre Raymond de Montmort (1678-1719) posed le probléme de rencontres: What is the probability that no 
matches take place in the game of rencontres? The solution to Montmort’s problem is the probability that a 
randomly selected permutation of 52 objects is a derangement, namely, D52/52!, which, as we will see, is 
approximately 1/e. 
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TABLE 1 The Probability of a Derangement. 


0.33333 


0.37500 


0.36667 0.36806 0.36786 


It is now simple to find D, for a given positive integer n. For instance, using Theorem 2, it 


follows that 


as we have previously remarked. 


The solution of the problem in Example 4 can now be given. 


Solution: The probability that no one receives the correct hat is D, /n!. By Theorem 2, this 


probability is 
Dn 1 
=] one LD". 
n! 1! 7 2! ea" n! 


The values of this probability for 2 < n < 7 are displayed in Table 1. 
Using methods from calculus it can be shown that 


Aa ts! 
e =l-—+— 
"2! 


1 
eet (-1D)"— +--+ © 0.368. 
! n! 


Because this is an alternating series with terms tending to zero, it follows that as n grows without 
bound, the probability that no one receives the correct hat converges to e~! ~ 0.368. In fact, 
this probability can be shown to be within 1/(n + 1)! of e7!. < 


Exercises 


1. Suppose that in a bushel of 100 apples there are 20 that 


have worms in them and 15 that have bruises. Only those 
apples with neither worms nor bruises can be sold. If there 
are 10 bruised apples that have worms in them, how many 
of the 100 apples can be sold? 


. Of 1000 applicants for a mountain-climbing trip in the 
Himalayas, 450 get altitude sickness, 622 are not in good 
enough shape, and 30 have allergies. An applicant qual- 
ifies if and only if this applicant does not get altitude 
sickness, is in good shape, and does not have allergies. If 
there are 111 applicants who get altitude sickness and are 
not in good enough shape, 14 who get altitude sickness 
and have allergies, 18 who are not in good enough shape 
and have allergies, and 9 who get altitude sickness, are 
not in good enough shape, and have allergies, how many 
applicants qualify? 

. How many solutions does the equation xj + x2 + +3 = 
13 have where x1, x2, and x3 are nonnegative integers less 
than 6? 


10. 


. Find the number of solutions of the equation x; + x2 + 


x3 + x4 = 17, where x;, i = 1, 2,3,4, are nonnegative 
integers such that x) < 3, x2 < 4, x3 <5, and x4 < 8. 


. Find the number of primes less than 200 using the prin- 


ciple of inclusion—exclusion. 


. An integer is called squarefree if it is not divisible by 


the square of a positive integer greater than 1. Find the 
number of squarefree positive integers less than 100. 


. How many positive integers less than 10,000 are not the 


second or higher power of an integer? 


. How many onto functions are there from a set with seven 


elements to one with five elements? 


. How many ways are there to distribute six different toys 


to three different children such that each child gets at least 
one toy? 
In how many ways can eight distinct balls be distributed 
into three distinct urns if each urn must contain at least 
one ball? 


11. In how many ways can seven different jobs be assigned 
to four different employees so that each employee is as- 
signed at least one job and the most difficult job is as- 
signed to the best employee? 


12. List all the derangements of {1, 2, 3, 4}. 


13. How many derangements are there of a set with seven 
elements? 


14. What is the probability that none of 10 people receives 
the correct hat if a hatcheck person hands their hats back 
randomly? 


15. A machine that inserts letters into envelopes goes haywire 
and inserts letters randomly into envelopes. What is the 
probability that in a group of 100 letters 
a) no letter is put into the correct envelope? 

b) exactly one letter is put into the correct envelope? 
c) exactly 98 letters are put into the correct envelopes? 
d) exactly 99 letters are put into the correct envelopes? 
e) all letters are put into the correct envelopes? 

16. A group of n students is assigned seats for each of two 
classes in the same classroom. How many ways can these 
seats be assigned if no student is assigned the same seat 
for both classes? 


*17,. How many ways can the digits 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 be 
arranged so that no even digit is in its original position? 


*18. Use a combinatorial argument to show that the sequence 
{D,}, where D, denotes the number of derangements 
of n objects, satisfies the recurrence relation 


Dy = (n — 1)(Dn—-1 + Dn—2) 


Key Terms and Results 


TERMS 


recurrence relation: a formula expressing terms of a se- 
quence, except for some initial terms, as a function of one 
or more previous terms of the sequence 

initial conditions for a recurrence relation: the values of the 
terms of a sequence satisfying the recurrence relation before 
this relation takes effect 

dynamic programming: an algorithmic paradigm that finds 
the solution to an optimization problem by recursively 
breaking down the problem into overlapping subproblems 
and combining their solutions with the help of a recurrence 
relation 

linear homogeneous recurrence relation with constant co- 
efficients: a recurrence relation that expresses the terms of 
a sequence, except initial terms, as a linear combination of 
previous terms 

characteristic roots of a linear homogeneous recurrence 
relation with constant coefficients: the roots of the poly- 
nomial associated with a linear homogeneous recurrence 
relation with constant coefficients 
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forn > 2. [Hint: Note that there are n — 1 choices for the 
first element & of a derangement. Consider separately the 
derangements that start with & that do and do not have 1 
in the kth position.] 


*19. Use Exercise 18 to show that 
Dn =nDn—1 + (-1)" 


forn > 1. 
20. Use Exercise 19 to find an explicit formula for Dy. 


21. For which positive integers n is D,, the number of de- 
rangements of n objects, even? 


22. Suppose that p and q are distinct primes. Use the prin- 
ciple of inclusion—exclusion to find ¢(pq), the number 
of positive integers not exceeding pq that are relatively 
prime to pq. 

* 23. Use the principle of inclusion—exclusion to derive a for- 
mula for ¢(n) when the prime factorization of n is 
a) dz ain 
N= Py Py ***Pm- 
*24, Show that if n is a positive integer, then 
n! = C(n,0)D, + C(n, 1) Dy-1 
+---+C(n,n —1)D, + C(n,n)Do, 


where Dy is the number of derangements of k objects. 
25. How many derangements of {1, 2, 3, 4, 5, 6} begin with 
the integers 1, 2, and 3, in some order? 


26. How many derangements of {1, 2, 3, 4, 5, 6} end with the 
integers 1, 2, and 3, in some order? 


27. Prove Theorem 1. 


linear nonhomogeneous recurrence relation with constant 
coefficients: a recurrence relation that expresses the terms 
of a sequence, except for initial terms, as a linear combina- 
tion of previous terms plus a function that is not identically 
zero that depends only on the index 

divide-and-conquer algorithm: an algorithm that solves a 
problem recursively by splitting it into a fixed number of 
smaller non-overlapping subproblems of the same type 

generating function of a sequence: the formal series that has 
the nth term of the sequence as the coefficient of x” 

sieve of Eratosthenes: a procedure for finding the primes less 
than a specified positive integer 

derangement: a permutation of objects such that no object is 
in its original place 


RESULTS 


the formula for the number of elements in the union of two 
finite sets: 


|AU B| = |A| + |B] -|ANB| 
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the formula for the number of elements in the union of three 


finite sets: 


|AUBUC|=|A|+|B]+|C|—|AN B|—|ANC| 


—|BNC|+|ANBNC| 


the principle of inclusion—exclusion: 


|A,UA2U++-UAnl = D> |AiI— DANA] 


l<i<n 


Pe 


l<i<j<k<n 


l<i<j<n 


|A; 1 Aj O Ag| 


me DNA AD --- An 


Review Questions 


1. 


a) What is a recurrence relation? 


b) Find a recurrence relation for the amount of money 
that will be in an account after n years if $1,000,000 
is deposited in an account yielding 9% annually. 


. Explain how the Fibonacci numbers are used to solve Fi- 


bonacci’s problem about rabbits. 


. a) Find a recurrence relation for the number of steps 


needed to solve the Tower of Hanoi puzzle. 


b) Show how this recurrence relation can be solved using 
iteration. 


. a) Explain how to find a recurrence relation for the num- 


ber of bit strings of length n not containing two con- 
secutive ls. 


b) Describe another counting problem that has a solution 
satisfying the same recurrence relation. 


. a) Whatis dynamic programming and how are recurrence 


relations used in algorithms that follow this paradigm? 


b) Explain how dynamic programming can be used to 
schedule talks in a lecture hall from a set of possible 
talks to maximize overall attendance. 


. Define a linear homogeneous recurrence relation of de- 


gree k. 


. a) Explain how to solve linear homogeneous recurrence 


relations of degree 2. 


b) Solve the recurrence relation ad, = 13d,_1) — 22ay,_2 
forn > 2 if ag = 3 and a; = 15. 


c) Solve the recurrence relation a, = 
forn > 2 if ag = 3 and a; = 35. 


14ay,_—1 — 49ay—2 


. a) Explain how to find f (b*) where k is a positive inte- 


gerif f (n) satisfies the divide-and-conquer recurrence 
relation f(n) = af (n/b) + g(n) whenever b divides 
the positive integer n. 

b) Find f(256) if f(n) =3f(n/4) +5n/4 
fQ) =7. 


and 


the number of onto functions from a set with m elements 


to a set with n elements: 


n” — C(n, 1)(n — 1)" + C(n, 2)(n — 2)” 
—-e)+(-1)""'!C(n,n—1)-1" 


the number of derangements of n objects: 


9. a) 


10. 


11. 


12. 


13. 


14 


Derive a divide-and-conquer recurrence relation for 
the number of comparisons used to find a number in 
a list using a binary search. 


b 


wm 


Give a big-O estimate for the number of comparisons 
used by a binary search from the divide-and-conquer 
recurrence relation you gave in (a) using Theorem 1 
in Section 8.3. 


a) Give a formula for the number of elements in the union 
of three sets. 


b 


c) Explain how to use the formula from (a) to find the 
number of integers not exceeding 1000 that are divis- 
ible by 6, 10, or 15. 


Explain how to use the formula from (a) to find the 
number of solutions in nonnegative integers to the 
equation x; + x2 + x3 + x4 = 22 with x; < 8, x2 < 
6, and x3 < 5. 


mm 


Explain why this formula is valid. 


d 


w~m 


a) Give a formula for the number of elements in the union 
of four sets and explain why it is valid. 


b 


~ 


Suppose the sets Aj, Az, A3, and A4 each contain 25 
elements, the intersection of any two of these sets con- 
tains 5 elements, the intersection of any three of these 
sets contains 2 elements, and | element is in all four 
of the sets. How many elements are in the union of the 
four sets? 


a) State the principle of inclusion—exclusion. 


b) Outline a proof of this principle. 


Explain how the principle of inclusion—exclusion can be 
used to count the number of onto functions from a set 
with m elements to a set with n elements. 


. a) How can you count the number of ways to assign m 
jobs ton employees so that each employee is assigned 
at least one job? 


b) How many ways are there to assign seven jobs to three 
employees so that each employee is assigned at least 
one job? 


15. Explain how the inclusion—-exclusion principle can be 


used to count the number of primes not exceeding the 
positive integer n. 


Supplementary Exercises 


1. A group of 10 people begin a chain letter, with each per- 


son sending the letter to four other people. Each of these 

people sends the letter to four additional people. 

a) Find a recurrence relation for the number of letters 
sent at the nth stage of this chain letter, if no person 
ever receives more than one letter. 

b) What are the initial conditions for the recurrence rela- 
tion in part (a)? 

c) How many letters are sent at the nth stage of the chain 
letter? 


2. A nuclear reactor has created 18 grams of a particular 


radioactive isotope. Every hour 1% of this radioactive 

isotope decays. 

a) Set up a recurrence relation for the amount of this 
isotope left n hours after its creation. 


b) What are the initial conditions for the recurrence rela- 
tion in part (a)? 

c) Solve this recurrence relation. 

. Every hour the U.S. government prints 10,000 more $1 

bills, 4000 more $5 bills, 3000 more $10 bills, 2500 more 

$20 bills, 1000 more $50 bills, and the same number of 

$100 bills as it did the previous hour. In the initial hour 

1000 of each bill were produced. 

a) Set up a recurrence relation for the amount of money 
produced in the nth hour. 

b) What are the initial conditions for the recurrence rela- 
tion in part (a)? 

c) Solve the recurrence relation for the amount of money 
produced in the mth hour. 

d) Set up a recurrence relation for the total amount of 
money produced in the first n hours. 

e) Solve the recurrence relation for the total amount of 
money produced in the first n hours. 

. Suppose that every hour there are two new bacteria in a 

colony for each bacterium that was present the previous 

hour, and that all bacteria 2 hours old die. The colony 

starts with 100 new bacteria. 

a) Set up a recurrence relation for the number of bacteria 
present after n hours. 


b) What is the solution of this recurrence relation? 

c) When will the colony contain more than 1 million bac- 
teria? 

. Messages are sent over a communications channel using 

two different signals. One signal requires 2 microseconds 


16. 


*10. 


11. 


12. 


* 13. 


*14, 
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a) Define a derangement. 

b) Why is counting the number of ways a hatcheck per- 
son can return hats to n people, so that no one receives 
the correct hat, the same as counting the number of 
derangements of n objects? 

c) Explain how to count the number of derangements of 
n objects. 


for transmittal, and the other signal requires 3 microsec- 

onds for transmittal. Each signal of a message is followed 

immediately by the next signal. 

a) Find a recurrence relation for the number of different 
signals that can be sent in n microseconds. 

b) What are the initial conditions of the recurrence rela- 
tion in part (a)? 

c) How many different messages can be sent in 12 mi- 
croseconds? 


. A small post office has only 4-cent stamps, 6-cent 


stamps, and 10-cent stamps. Find a recurrence relation 
for the number of ways to form postage of n cents with 
these stamps if the order that the stamps are used mat- 
ters. What are the initial conditions for this recurrence 
relation? 


. How many ways are there to form these postages using 


the rules described in Exercise 6? 


b) 14 cents 
d) 22 cents 


a) 12 cents 
c) 18 cents 


. Find the solutions of the simultaneous system of recur- 


rence relations 
An = An—1 + bn 
by = Gn—1 — Dn-1 


with ag = 1 and bo = 2. 


. Solve the recurrence relation a, = a4 /an—2 1f ag = 1 


and a, = 2. [Hint: Take logarithms of both sides to 
obtain a recurrence relation for the sequence log dy, 
n=0,1,2,....] 

Solve the recurrence relation a, = Ce ae if ag = 2 
and a, = 2. (See the hint for Exercise 9.) 

Find the solution of the recurrence relation a, = 
3an—1 — 3dn—2 + Qn—3 + 1 if ag = 2, ay = 4, and a2 = 
8. 

Find the solution of the recurrence relation a, 
= 3ay,_1 — 3ay_-2 + an—3 if ag = 2, ay = 2, and az = 4. 


Suppose that in Example 1 of Section 8.1 a pair of rabbits 
leaves the island after reproducing twice. Find a recur- 
rence relation for the number of rabbits on the island in 
the middle of the nth month. 


In this exercise we construct a dynamic programming al- 
gorithm for solving the problem of finding a subset S 
of items chosen from a set of n items where item i has 
a weight w;, which is a positive integer, so that the to- 
tal weight of the items in S is a maximum but does not 


568 8/ Advanced Counting Techniques 


exceed a fixed weight limit W. Let M(j, w) denote the 

maximum total weight of the items in a subset of the 

first j items such that this total weight does not exceed w. 

This problem is known as the knapsack problem. 

a) Show that if w; > w, then M(j,w) = M(j — 1, w). 

b) Show that if wj;<w, then M(j,w)= 
max(M(j — 1,w),w; + M(j —1,w—wj)). 

c) Use (a) and (b) to construct a dynamic programming 
algorithm for determining the maximum total weight 
of items so that this total weight does not exceed W. 
In your algorithm store the values M(j, w) as they are 
found. 

d) Explain how you can use the values M(j, w) com- 
puted by the algorithm in part (c) to find a subset of 
items with maximum total weight not exceeding W. 


In Exercises 15—18 we develop a dynamic programming al- 
gorithm for finding a longest common subsequence of two 


sequences dj, d2,.. 


-,@m and by, b2,..., by, an important 


problem in the comparison of DNA of different organisms. 


15. 


16. 


17. 


18. 


19, 


20. 


21. 


Suppose that cj,c2,...,Cp is a longest common 
subsequence of the sequences aj, d2,.. and 
by, bo, ..., by. 

a) Show that if adm —=bnj, then cp =am = by, and 
C}, C2,..., Cp—1 18 a longest common subsequence of 
a1, 42,..-,4m—, and by, bz, ..., bn, when p > 1. 

b) Suppose that dm 4 bn. Show that if cp 4am, 


->Amn 


then cj,C2,...,Cp is a longest common. subse- 
quence of a1,42,...,@m—, and bj, bo,...,b, and 
also show that if cp # by, then cj,c2,...,Cp is a 
longest common subsequence of aj, a2,...,@m and 


by, bo, ..., Dn—1. 

Let L(i, j) denote the length of a longest com- 
mon subsequence of a, 42,...,a; and bj, b2,..., bj, 
where 0 <i <m and 0 < j <n. Use parts (a) and (b) 
of Exercise 15 to show that L(i, j) satisfies the recur- 
rence relation L(i, j) = LG —1, 7 —1)+1 if both i 
and j are nonzero and aj=b;, and L(i, j)= 
max(L(i, j — 1), L@ — 1, 7)) if bothi and j are nonzero 
anda; # b;, and the initial condition L(i, 7) = Oifi = 0 
or j = 0. 

Use Exercise 16 to construct a dynamic programming 
algorithm for computing the length of a longest com- 
mon subsequence of two sequences a1, a2,...,@m and 


bi, bo,..., by, storing the values of L(i, j) as they are 
found. 
Develop an algorithm for finding a longest com- 


mon subsequence of two sequences dj, a2,...,@m and 
bi, b2,..., by using the values L(i, 7) found by the algo- 
rithm in Exercise 17. 

Find the solution to the recurrence relation f(n) = 
f(n/2) + n? for n = 2* where k is a positive integer and 
fM= lL. 

Find the solution to the recurrence relation f(n) = 
3 f (n/5) +2n*, when n is divisible by 5, for n = 5*, 
where k is a positive integer and f(1) = 1. 

Give a big-O estimate for the size of f in Exercise 20 
if f is an increasing function. 


22. 


23. 


24. A sequence aj, a2,.. 


25. 


Find a recurrence relation that describes the number of 
comparisons used by the following algorithm: Find the 
largest and second largest elements of a sequence of n 
numbers recursively by splitting the sequence into two 
subsequences with an equal number of terms, or where 
there is one more term in one subsequence than in the 
other, at each stage. Stop when subsequences with two 
terms are reached. 


Give a big-O estimate for the number of comparisons 
used by the algorithm described in Exercise 22. 


-,@, 1S unimodal if and only if 
there is an index m, 1 <m <n, such that aj < aj41 
when | <i <™m and aj > aj41 when m <i <n. That 
is, the terms of the sequence strictly increase until the 
mth term and they strictly decrease after it, which implies 
that a, is the largest term. In this exercise, ad, will al- 
ways denote the largest term of the unimodal sequence 
a1, 42,..., 4p. 
a) Show that a,, is the unique term of the sequence that 
is greater than both the term immediately preceding it 
and the term immediately following it. 


b) Show that if aj <aj4,; where 1<i<n, 
theni+1<m <n. 
c) Show that if aj; >aj4,; where 1 <i <n, 


then 1 < m <i. 


d) Develop a divide-and-conquer algorithm for locat- 
ing the index m. [Hint: Suppose that i <m < j. 
Use parts (a), (b), and (c) to determine whether 
LG+j)/2)+1<m<n, l1<m< [G+ J)/2] -1, 
orm = |@ + j)/2).] 

Show that the algorithm from Exercise 24 has worst-case 

time complexity O (log 1) in terms of the number of com- 

parisons. 


Let {a,} be a sequence of real numbers. The forward dif- 
ferences of this sequence are defined recursively as fol- 
lows: The first forward difference is Aa, = a,+, — an; the 
(k + 1)st forward difference A‘+!a,, is obtained from A*ay 
by Atl, = Meany — May. 


26. 


27. 


* 28. 


29. 


30. 


Find Aa,, where 

a) dy =3. b) dy =4n+7. ©) dy=n?+n4+1. 

Let a, = 3n3 +n +2. Find Man, where k equals 

a) 2. b) 3. c) 4. 

Suppose that a, = P(n), where P is a polynomial of de- 


gree d. Prove that A¢+!a, = 0 for all nonnegative inte- 
gers n. 


Let {a,} and {b,} be sequences of real numbers. Show 
that 


A(anbn) = an41(Abn) + bn (Aan). 


Show that if F(x) and G(x) are the generating func- 
tions for the sequences {ax} and {b,x}, respectively, 
and c and d are real numbers, then (cF + dG )(x) is the 
generating function for {cag + dbx}. 


31. (Requires calculus) This exercise shows how generating 

functions can be used to solve the recurrence relation 

(n + 1)dyn41 = a, + (1/n!) for n > O with initial condi- 

tion ag = 1. 

a) Let G(x) be the generating function for {a,}. Show 
that G’(x) = G(x) + e* and G(0) = 1. 

b) Show from part (a) that (e~* G(x))’ = 1, and conclude 
that G(x) = xe* + e*. 

c) Use part (b) to find a closed form for a,. 


32. Suppose that 14 students receive an A on the first exam 
in a discrete mathematics class, and 18 receive an A on 
the second exam. If 22 students received an A on either 
the first exam or the second exam, how many students 
received an A on both exams? 


33. There are 323 farms in Monmouth County that have at 
least one of horses, cows, and sheep. If 224 have horses, 
85 have cows, 57 have sheep, and 18 farms have all three 
types of animals, how many farms have exactly two of 
these three types of animals? 


34. Queries to a database of student records at a college pro- 
duced the following data: There are 2175 students at the 
college, 1675 of these are not freshmen, 1074 students 
have taken a course in calculus, 444 students have taken a 
course in discrete mathematics, 607 students are not fresh- 
men and have taken calculus, 350 students have taken 
calculus and discrete mathematics, 201 students are not 
freshmen and have taken discrete mathematics, and 143 
students are not freshmen and have taken both calculus 
and discrete mathematics. Can all the responses to the 
queries be correct? 


35. Students in the school of mathematics at a university ma- 
jor in one or more of the following four areas: applied 
mathematics (AM), pure mathematics (PM), operations 
research (OR), and computer science (CS). How many 
students are in this school if (including joint majors) there 


Computer Projects 


Write programs with these input and output. 


1. Given a positive integer n, list all the moves required in 
the Tower of Hanoi puzzle to move n disks from one peg 
to another according to the rules of the puzzle. 


2. Given a positive integer n and an integerk with] <k <n, 
list all the moves used by the Frame—Stewart algorithm 
(described in the preamble to Exercise 38 of Section 8.1) 
to move n disks from one peg to another using four pegs 
according to the rules of the puzzle. 


3. Given a positive integer n, list all the bit sequences of 
length n that do not have a pair of consecutive Os. 

4. Given an integer n greater than 1, write out all ways to 
parenthesize the product of n + 1 variables. 


5. Given a set of n talks, their start and end times, and the 
number of attendees at each talk, use dynamic program- 


36. 


37. 


38. 


39. 


* 40. 


41. 


42. 


43. 
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are 23 students majoring in AM; 17 in PM; 44 in OR; 63 
in CS; 5 in AM and PM; 8 in AM and CS; 4 in AM and 
OR; 6 in PM and CS; 5 in PM and OR; 14 in OR and CS; 
2 in PM, OR, and CS; 2 in AM, OR, and CS; | in PM, 
AM, and OR; | in PM, AM, and CS; and 1 in all four 
fields. 


How many terms are needed when the inclusion— 
exclusion principle is used to express the number of ele- 
ments in the union of seven sets if no more than five of 
these sets have a common element? 


How many solutions in positive integers are there 
to the equation x; +x2++x3=20 with 2 < x, <6, 
6 < x2 < 10, and0 < x3 < 5? 

How many positive integers less than 1,000,000 are 

a) divisible by 2, 3, or 5? 

b) not divisible by 7, 11, or 13? 

c) divisible by 3 but not by 7? 

How many positive integers less than 200 are 


a) second or higher powers of integers? 

b) either primes or second or higher powers of integers? 

c) not divisible by the square of an integer greater 
than 1? 

d) not divisible by the cube of an integer greater than 1? 

e) not divisible by three or more primes? 


How many ways are there to assign six different jobs to 
three different employees if the hardest job is assigned 
to the most experienced employee and the easiest job is 
assigned to the least experienced employee? 

What is the probability that exactly one person is given 
back the correct hat by a hatcheck person who gives n 
people their hats back at random? 

How many bit strings of length six do not contain four 
consecutive 1s? 

What is the probability that a bit string of length six cho- 
sen at random contains at least four 1s? 


ming to schedule a subset of these talks in a single lecture 
hall to maximize total attendance. 


. Given matrices Ay, Az,...,A,, with dimensions m, x 


m2z,M2 X M3,...,Mp X My+1, Tespectively, each with 
integer entries, use dynamic programming, as out- 
lined in Exercise 57 in Section 8.1, to find the mini- 
mum number of multiplications of integers needed to 
compute AyAz--- Aj). 


. Given arecurrence relation a, = Cjdy,—1 + C2dn—2, where 


cy, and cz are real numbers, initial conditions ag = Co and 
a, = Ci, and a positive integer k, find ax using iteration. 


. Given a recurrence relation dy = cjda,—1 + c2dn—2 and 


initial conditions ag = Co and a; = Cy, determine the 
unique solution. 
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9. 


10. 


Given a recurrence relation of the form f(n) = 
af (n/b) +c, where a is a real number, b is a positive 
integer, and c is a real number, and a positive integer k, 
find f (b*) using iteration. 

Given the number of elements in the intersection of three 
sets, the number of elements in each pairwise intersection 
of these sets, and the number of elements in each set, find 
the number of elements in their union. 


Computations and Explorations 


11. 


12. 


13. 


Given a positive integer n, produce the formula for the 
number of elements in the union of 7 sets. 


Given positive integers m and n, find the number of onto 
functions from a set with m elements to a set with n ele- 
ments. 


Given a positive integer n, list all the derangements of the 
set {1,2,3,..., n}. 


Use a computational program or programs you have written to do these exercises. 


. Find the exact value of fioo, fsoo, and fiooo, where f;, is 


the nth Fibonacci number. 


. Find the smallest Fibonacci number greater than 


1,000,000, greater than 1,000,000,000, and greater than 
1,000,000,000,000. 


. Find as many prime Fibonacci numbers as you can. It is 


unknown whether there are infinitely many of these. 


. Write out all the moves required to solve the Tower of 


Hanoi puzzle with 10 disks. 


. Write out all the moves required to use the Frame—Stewart 


algorithm to move 20 disks from one peg to another peg 
using four pegs according to the rules of the Reve’s puz- 
ale. 


. Verify the Frame conjecture for solving the Reve’s puzzle 


for n disks for as many integers n as possible by show- 
ing that the puzzle cannot be solved using fewer moves 
than are made by the Frame—Stewart algorithm with the 
optimal choice of k. 


Writing Projects 


Respond to these with essays using outside sources. 


1. Find the original source where Fibonacci presented his 


puzzle about modeling rabbit populations. Discuss this 
problem and other problems posed by Fibonacci and give 
some information about Fibonacci himself. 


. Explain how the Fibonacci numbers arise in a variety of 


applications, such as in phyllotaxis, the study of arrange- 
ment of leaves in plants, in the study of reflections by 
mirrors, and so on. 


. Describe different variations of the Tower of Hanoi puz- 


zle, including those with more than three pegs (includ- 
ing the Reve’s puzzle discussed in the text and exer- 
cises), those where disk moves are restricted, and those 
where disks may have the same size. Include what is 
known about the number of moves required to solve 
each variation. 


10. 
11. 


. Compute the number of operations required to multi- 


ply two integers with n bits for various integers n in- 
cluding 16, 64, 256, and 1024 using the fast multiplica- 
tion described in Example 4 of Section 8.3 and the stan- 
dard algorithm for multiplying integers (Algorithm 3 in 
Section 4.2). 


. Compute the number of operations required to multiply 


twon x n matrices for various integers n including 4, 16, 
64, and 128 using the fast matrix multiplication described 
in Example 5 of Section 8.3 and the standard algorithm 
for multiplying matrices (Algorithm 1 in Section 3.3). 


. Find the number of primes not exceeding 10,000 using 


the method described in Section 8.6 to find the number of 
primes not exceeding 100. 

List all the derangements of {1, 2, 3, 4, 5, 6, 7, 8}. 
Compute the probability that a permutation of n objects 
is a derangement for all positive integers not exceeding 
20 and determine how quickly these probabilities ap- 
proach the number 1/e. 


. Discuss as many different problems as possible where the 


Catalan numbers arise. 


. Discuss some of the problems in which Richard Bellman 


first used dynamic programming. 


. Describe the role dynamic programming algorithms play 


in bioinformatics including for DNA sequence compari- 
son, gene comparison, and RNA structure prediction. 


. Describe the use of dynamic programming in economics 


including its use to study optimal consumption and sav- 
ing. 


. Explain how dynamic programming can be used to solve 


the egg-dropping puzzle which determines from which 
floors of a multistory building it is safe to drop eggs from 
without breaking. 


10. 


11. 


12. 


. Describe the solution of Ulam’s problem (see Exercise 28 


in Section 8.3) involving searching with one lie found by 
Andrzej Pelc. 


Discuss variations of Ulam’s problem (see Exercise 28 in 
Section 8.3) involving searching with more than one lie 
and what is known about this problem. 


Define the convex hull of a set of points in the plane and 
describe three different algorithms, including a divide- 
and-conquer algorithm, for finding the convex hull of a 
set of points in the plane. 


Describe how sieve methods are used in number theory. 
What kind of results have been established using such 
methods? 


13. 


14. 


15. 


16. 


17. 
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Look up the rules of the old French card game of rencon- 
tres. Describe these rules and describe the work of Pierre 
Raymond de Montmort on le probléme de rencontres. 


Describe how exponential generating functions can be 
used to solve a variety of counting problems. 

Describe the Polya theory of counting and the kind of 
counting problems that can be solved using this theory. 


The probléme des ménages (the problem of the house- 
holds) asks for the number of ways to arrange n couples 
around a table so that the sexes alternate and no husband 
and wife are seated together. Explain the method used by 
E. Lucas to solve this problem. 

Explain how rook polynomials can be used to solve count- 
ing problems. 


9.1 Relations and 
Their 
Properties 


9.2. n-ary Relations 
and Their 
Applications 


9.3 Representing 
Relations 


9.4 Closures of 
Relations 


9.5 Equivalence 
Relations 


9.6 Partial 
Orderings 


Relations 


R vviostie between elements of sets occur in many contexts. Every day we deal with 
relationships such as those between a business and its telephone number, an employee 
and his or her salary, a person and a relative, and so on. In mathematics we study relationships 
such as those between a positive integer and one that it divides, an integer and one that it is 
congruent to modulo 5, a real number and one that is larger than it, a real number x and the 
value f(x) where f is a function, and so on. Relationships such as that between a program and 
a variable it uses, and that between a computer language and a valid statement in this language 
often arise in computer science. 

Relationships between elements of sets are represented using the structure called a relation, 
which is just a subset of the Cartesian product of the sets. Relations can be used to solve problems 
such as determining which pairs of cities are linked by airline flights in a network, finding a 
viable order for the different phases of a complicated project, or producing a useful way to store 
information in computer databases. 

In some computer languages, only the first 31 characters of the name of a variable matter. 
The relation consisting of ordered pairs of strings where the first string has the same initial 
31 characters as the second string is an example of a special type of relation, known as an 
equivalence relation. Equivalence relations arise throughout mathematics and computer science. 
We will study equivalence relations, and other special types of relations, in this chapter. 


Relations and Their Properties 


Links 


DEFINITION 1 


EXAMPLE 1 


Introduction 


The most direct way to express a relationship between elements of two sets is to use ordered pairs 
made up of two related elements. For this reason, sets of ordered pairs are called binary relations. 
In this section we introduce the basic terminology used to describe binary relations. Later in this 
chapter we will use relations to solve problems involving communications networks, project 
scheduling, and identifying elements in sets with common properties. 


Let A and B be sets. A binary relation from A to B is a subset of A x B. 


In other words, a binary relation from A to B is a set R of ordered pairs where the first element 
of each ordered pair comes from A and the second element comes from B. We use the notation 
a Rb to denote that (a,b) € R and a Rb to denote that (a, b) ¢ R. Moreover, when (a, b) 
belongs to R, a is said to be related to b by R. 

Binary relations represent relationships between the elements of two sets. We will introduce 
n-ary relations, which express relationships among elements of more than two sets, later in this 
chapter. We will omit the word binary when there is no danger of confusion. 

Examples 1-3 illustrate the notion of a relation. 


Let A be the set of students in your school, and let B be the set of courses. Let R be 
the relation that consists of those pairs (a,b), where a is a student enrolled in course b. 
For instance, if Jason Goodfriend and Deborah Sherman are enrolled in CS518, the pairs 
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EXAMPLE 2 


EXAMPLE 3 


(Jason Goodfriend, CS518) and (Deborah Sherman, CS518) belong to R. If Jason Goodfriend 
is also enrolled in CS510, then the pair (Jason Goodfriend, CS510) is also in R. However, 
if Deborah Sherman is not enrolled in CS510, then the pair (Deborah Sherman, CS510) is 
not in R. 

Note that if a student is not currently enrolled in any courses there will be no pairs in R that 
have this student as the first element. Similarly, if a course is not currently being offered there 
will be no pairs in R that have this course as their second element. < 


Let A be the set of cities in the U.S.A., and let B be the set of the 50 states in the U.S.A. 
Define the relation R by specifying that (a,b) belongs to R if a city with name a is in 
the state b. For instance, (Boulder, Colorado), (Bangor, Maine), (Ann Arbor, Michigan), 
(Middletown, New Jersey), (Middletown, New York), (Cupertino, California), and 
(Red Bank, New Jersey) are in R. < 


Let A = {0, 1, 2} and B = {a, b}. Then {(0, a), (0, b), (1, a), (2, b)} is a relation from A to B. 
This means, for instance, that 0 Ra, but that 1 R b. Relations can be represented graphically, 
as shown in Figure 1, using arrows to represent ordered pairs. Another way to represent this 
relation is to use a table, which is also done in Figure 1. We will discuss representations of 
relations in more detail in Section 9.3. 


Oe 
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FIGURE 1 _ Displaying the Ordered Pairs in the Relation R from Example 3. 


Functions as Relations 


Recall that a function f from a set A to a set B (as defined in Section 2.3) assigns exactly 
one element of B to each element of A. The graph of f is the set of ordered pairs (a, b) such 
that b = f(a). Because the graph of f is a subset of A x B, it is a relation from A to B. 
Moreover, the graph of a function has the property that every element of A is the first element 
of exactly one ordered pair of the graph. 

Conversely, if R is a relation from A to B such that every element in A is the first element 
of exactly one ordered pair of R, then a function can be defined with R as its graph. This can be 
done by assigning to an element a of A the unique element b € B such that (a, b) € R. (Note 
that the relation R in Example 2 is not the graph of a function because Middletown occurs more 
than once as the first element of an ordered pair in R.) 

A relation can be used to express a one-to-many relationship between the elements of the 
sets A and B (as in Example 2), where an element of A may be related to more than one element 
of B. A function represents a relation where exactly one element of B is related to each element 
of A. 

Relations are a generalization of graphs of functions; they can be used to express a much 
wider class of relationships between sets. (Recall that the graph of the function f from A to B 
is the set of ordered pairs (a, f(a)) fora € A.) 


DEFINITION 2 


EXAMPLE 4 


EXAMPLE 5 
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Relations on a Set 


Relations from a set A to itself are of special interest. 


A relation on a set A is a relation from A to A. 


In other words, a relation on a set A is a subset of A x A. 


Let A be the set {1, 2, 3, 4}. Which ordered pairs are in the relation R = {(a, b) | a divides b}? 


Solution: Because (a, b) is in R if and only if a and b are positive integers not exceeding 4 such 
that a divides b, we see that 


R={(1; 1), 2),G;3), 0,4), 2.2), 2.4), G,3),4, D), 
The pairs in this relation are displayed both graphically and in tabular form in Figure 2. < 


Next, some examples of relations on the set of integers will be given in Example 5. 


Consider these relations on the set of integers: 


Ri = {(a,b)|a < B}, 

Ro = {(a, b) | a > b}, 

R3 = {(a,b) |a=bora=-—b}, 
R4 = {(a,b) |a =D}, 

Rs = {(a,b) |a=b+ }}, 

Ro = {(a, b) |a+b < 3}. 


Which of these relations contain each of the pairs (1, 1), 1, 2), (2, 1), (1, —1), and (2, 2)? 


Remark: Unlike the relations in Examples 1-4, these are relations on an infinite set. 


Solution: The pair (1, 1) is in R}, R3, Ra, and Ro; (1, 2) is in Ry and Reo; (2, 1) is in Ro, Rs, 
and Re; (1, —1) is in Ro, R3, and R6; and finally, (2, 2) is in Rj, R3, and Ra. < 


It is not hard to determine the number of relations on a finite set, because a relation on a 
set A is simply a subset of A x A. 
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FIGURE 2 Displaying the Ordered Pairs in 
the Relation R from Example 4. 
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EXAMPLE 6 


DEFINITION 3 


EXAMPLE 7 


EXAMPLE 8 


EXAMPLE 9 


How many relations are there on a set with n elements? 


Solution: A relation on a set A is a subset of A x A. Because A x A has n? elements when A 
has n elements, and a set with m elements has 2” subsets, there are 2”. subsets of A x A. Thus, 
there are 2”” relations ona set with n elements. For example, there are 23° = 29 = 512 relations 
on the set {a, b, c}. < 


Properties of Relations 


There are several properties that are used to classify relations on a set. We will introduce the 
most important of these here. 

In some relations an element is always related to itself. For instance, let R be the relation 
on the set of all people consisting of pairs (x, y) where x and y have the same mother and the 
same father. Then x Rx for every person x. 


A relation R on a set A is called reflexive if (a, a) € R for every element a € A. 


Remark: Using quantifiers we see that the relation R on the set A is reflexive if Va((a, a) € R), 
where the universe of discourse is the set of all elements in A. 


We see that a relation on A is reflexive if every element of A is related to itself. 
Examples 7—9 illustrate the concept of a reflexive relation. 


Consider the following relations on {1, 2, 3, 4}: 


R; = {(1, 1), (, 2), @, D, @, 2), G, 4), 4, D, 4,4}, 

Ry = {(1, 1), 1, 2), 2, D}, 

R3 = {(1, 1), (1, 2), (1, 4), @, D, @, 2), G, 3), 4, D, 4, OD}, 

Rg = {(Q2, 1), G, 1), G, 2), 4, D, G, 2), 4, 3}, 

Rs = {(1, 1), (1, 2), (1, 3), (1, 4), (2, 2), (2, 3), 2,4), 3, 3), 3,4, 4,9}, 
Re = {3, 4}. 


Which of these relations are reflexive? 


Solution: The relations R3 and Rs are reflexive because they both contain all pairs of the form 
(a, a), namely, (1, 1), (2, 2), (3, 3), and (4, 4). The other relations are not reflexive because 
they do not contain all of these ordered pairs. In particular, Rj, Ro, R4, and R6 are not reflexive 
because (3, 3) is not in any of these relations. < 


Which of the relations from Example 5 are reflexive? 


Solution: The reflexive relations from Example 5 are R; (because a < a for every integer a), 
R3, and R4. For each of the other relations in this example it is easy to find a pair of the 
form (a, a) that is not in the relation. (This is left as an exercise for the reader.) < 


Is the “divides” relation on the set of positive integers reflexive? 


Solution: Because a | a whenever a is a positive integer, the “divides” relation is reflexive. (Note 
that if we replace the set of positive integers with the set of all integers the relation is not reflexive 
because by definition 0 does not divide 0.) < 


DEFINITION 4 


EXAMPLE 10 


Extra p> 
Examples 


EXAMPLE 11 
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In some relations an element is related to a second element if and only if the second element 
is also related to the first element. The relation consisting of pairs (x, y), where x and y are 
students at your school with at least one common class has this property. Other relations have 
the property that if an element is related to a second element, then this second element is not 
related to the first. The relation consisting of the pairs (x, y), where x and y are students at your 
school, where x has a higher grade point average than y has this property. 


Arelation R onaset A is called symmetric if (b, a) € R whenever (a, b) € R,foralla,b € A. 
A relation R on a set A such that for all a, b € A, if (a,b) € R and (b,a) € R, thena = b 
is called antisymmetric. 


Remark: Using quantifiers, we see that the relation R on the set A is symmetric if 
VaVb((a, b) € R > (b, a) € R). Similarly, the relation R on the set A is antisymmetric if 
VaVb(((a, b) € RA (b,a) € R) > (a =b)). 


That is, a relation is symmetric if and only if a is related to b implies that b is related to a. 
A relation is antisymmetric if and only if there are no pairs of distinct elements a and b with a 
related to b and b related to a. That is, the only way to have a related to b and b related to a is 
for a and b to be the same element. The terms symmetric and antisymmetric are not opposites, 
because a relation can have both of these properties or may lack both of them (see Exercise 
10). A relation cannot be both symmetric and antisymmetric if it contains some pair of the form 
(a,b), where a # b. 


‘ 2 ; : : 
Remark: Although relatively few of the 2” relations on a set with n elements are symmetric 
or antisymmetric, as counting arguments can show, many important relations have one of these 
properties. (See Exercise 47.) 


Which of the relations from Example 7 are symmetric and which are antisymmetric? 


Solution: The relations R2 and R3 are symmetric, because in each case (b, a) belongs to the 
relation whenever (a, b) does. For R2, the only thing to check is that both (2, 1) and (1, 2) are 
in the relation. For R3, it is necessary to check that both (1, 2) and (2, 1) belong to the relation, 
and (1,4) and (4, 1) belong to the relation. The reader should verify that none of the other 
relations is symmetric. This is done by finding a pair (a, b) such that it is in the relation 
but (b, a) is not. 

R4, Rs, and R¢ are all antisymmetric. For each of these relations there is no pair of elements 
a and b with a € b such that both (a, b) and (b, a) belong to the relation. The reader should 
verify that none of the other relations is antisymmetric. This is done by finding a pair (a, b) 
with a £ b such that (a, b) and (b, a) are both in the relation. 


Which of the relations from Example 5 are symmetric and which are antisymmetric? 


Solution: The relations R3, R4, and Re are symmetric. R3 is symmetric, for ifa = bora = —b, 
then b =a or b= ~—a. Rgq is symmetric because a = b implies that b = a. Re is symmetric 
because a + b < 3 implies that b+ a < 3. The reader should verify that none of the other 
relations is symmetric. 

The relations R,, Ro, R4, and R5 are antisymmetric. R; is antisymmetric because the 
inequalities a < b and b < a imply that a = b. R> is antisymmetric because it is impossible 
that a > b and b >a. Rg is antisymmetric, because two elements are related with respect to 
Ra, if and only if they are equal. Rs is antisymmetric because it is impossible that a = b+ | 
and b = a + 1. The reader should verify that none of the other relations is antisymmetric. <4 
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EXAMPLE 12 


DEFINITION 5 


EXAMPLE 13 


Extra p> 
Examples a 


EXAMPLE 14 


EXAMPLE 15 


EXAMPLE 16 


Is the “divides” relation on the set of positive integers symmetric? Is it antisymmetric? 


Solution: This relation is not symmetric because 1|2, but 2 J 1. It is antisymmetric, for if a 
and b are positive integers with a|b and b|a, then a = b (the verification of this is left as an 
exercise for the reader). 


Let R be the relation consisting of all pairs (x, y) of students at your school, where x has 
taken more credits than y. Suppose that x is related to y and y is related to z. This means 
that x has taken more credits than y and y has taken more credits than z. We can conclude 
that x has taken more credits than z, so that x is related to z. What we have shown is that R has 
the transitive property, which is defined as follows. 


A relation R on a set A is called transitive if whenever (a,b) € R and (b,c) € R, 
then (a,c) € R, for alla, b,c € A. 


Remark: Using quantifiers we see that the relation R on a set A is transitive if we have 
VaVbVc(((a, b) € RA (b,c) € R) > (a,c) € R). 


Which of the relations in Example 7 are transitive? 


Solution: R4, Rs, and R¢ are transitive. For each of these relations, we can show that it is 
transitive by verifying that if (a, b) and (b, c) belong to this relation, then (a, c) also does. For 
instance, Ry is transitive, because (3, 2) and (2, 1), (4, 2) and (2, 1), (4, 3) and (3, 1), and (4, 3) 
and (3, 2) are the only such sets of pairs, and (3, 1), (4, 1), and (4, 2) belong to Ra. The reader 
should verify that Rs and R¢ are transitive. 

R, is not transitive because (3,4) and (4,1) belong to Rj, but (3, 1) does not. Ro is 
not transitive because (2, 1) and (1, 2) belong to R2, but (2, 2) does not. R3 is not transitive 
because (4, 1) and (1, 2) belong to R3, but (4, 2) does not. < 


Which of the relations in Example 5 are transitive? 


Solution: The relations R;, Ro, R3, and R4 are transitive. R; is transitive becausea < bandb <c 
imply that a < c. R2 is transitive because a > b and b > c imply that a > c. R3 is transitive 
because a = +b and b = +c imply that a = +c. Rj is clearly transitive, as the reader should 
verify. Rs is not transitive because (2, 1) and (1, 0) belong to Rs, but (2, 0) does not. Reg is not 
transitive because (2, 1) and (1, 2) belong to Re, but (2, 2) does not. < 


Is the “divides” relation on the set of positive integers transitive? 


Solution: Suppose that a divides b and b divides c. Then there are positive integers k and / 
such that b = ak and c = Dl. Hence, c = a(kl), so a divides c. It follows that this relation is 
transitive. < 


We can use counting techniques to determine the number of relations with specific proper- 
ties. Finding the number of relations with a particular property provides information about how 
common this property is in the set of all relations on a set with n elements. 


How many reflexive relations are there on a set with n elements? 
Solution: A relation R on a set A is a subset of A x A. Consequently, a relation is determined 


by specifying whether each of the n” ordered pairs in A x A is in R. However, if R is reflexive, 
each of the n ordered pairs (a, a) for a € A must be in R. Each of the other n(n — 1) ordered 


EXAMPLE 17 


EXAMPLE 18 


EXAMPLE 19 
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pairs of the form (a, b), where a 4 b, may or may not be in R. Hence, by the product rule for 
counting, there are 2”“"—!) reflexive relations [this is the number of ways to choose whether 
each element (a, b), with a # b, belongs to R]. 


Formulas for the number of symmetric relations and the number of antisymmetric re- 
lations on a set with n elements can be found using reasoning similar to that in Example 16 
(see Exercise 47). However, no general formula is known that counts the transitive relations on a 
set with n elements. Currently, T (n), the number of transitive relations on a set with n elements, is 
known only for n < 17. For example, T (4) = 3,994, T(5) = 154,303, and T (6) = 9,415,189. 


Combining Relations 


Because relations from A to B are subsets of A x B, two relations from A to B can be combined 
in any way two sets can be combined. Consider Examples 17-19. 


Let A=({1,2,3} and B=({1,2,3,4}. The relations R; = {(1,1), (2,2), (3,3)} and 
Ry = {C, 1), C1, 2), C1, 3), C1, 4)} can be combined to obtain 


Ri U Ro = {(1, )), C, 2), 0, 3), 1, 9), @, 2), G, 3D}, 
Ri Ro = {(1, D}, 

R, — Rp = {(2, 2), G, 3)}, 

Ro — Ri = {(1, 2), 1, 3), 1, 4}. 


< 


Let A and B be the set of all students and the set of all courses at a school, respectively. 
Suppose that R; consists of all ordered pairs (a, b), where a is a student who has taken course b, 
and R» consists of all ordered pairs (a, b), where a is a student who requires course b to graduate. 
What are the relations Rj U Ro, Rj N Ro, Rj ® Ro, Ri — Ro, and R2 — Rj? 


Solution: The relation R; U R2 consists of all ordered pairs (a, b), where a is a student who 
either has taken course b or needs course b to graduate, and R; M R2 is the set of all ordered 
pairs (a, b), where a is a student who has taken course b and needs this course to graduate. 
Also, R; ® R2 consists of all ordered pairs (a, b), where student a has taken course b but does 
not need it to graduate or needs course b to graduate but has not taken it. Rj — R2 is the set of 
ordered pairs (a, b), where a has taken course b but does not need it to graduate; that is, D is 
an elective course that a has taken. Ro — Rj is the set of all ordered pairs (a, b), where b is a 
course that a needs to graduate but has not taken. 


Let R, be the “less than” relation on the set of real numbers and let R2 be the “greater than” 
relation on the set of real numbers, that is, Ry = {(x, y) | x < y} and Ro = {(x, y) | x > y}. 
What are Ry; U Ro, Rt ON Ro, Ry — Ro, Ro — Rj, and Rj @ Ro? 


Solution: We note that (x, y) € Rj U Ro if and only if (x, y) € Ry or (x, y) € Ro. Hence, 
(x, y) € Rj U Ro if and only if x < y or x > y. Because the condition x < y or x > y is 
the same as the condition x  y, it follows that Rj U R2 = {(x, y) | x # y}. In other words, the 
union of the “less than” relation and the “greater than” relation is the “not equals” relation. 
Next, note that it is impossible for a pair (x, y) to belong to both R; and R2 because it is 
impossible that x < y andx > y. It follows that Rj N R2 = Y. We also see that Rj — Ro = Rj, 
Ry — Ry = Ro, and Ry ® Ry = Ry U Ry — RIN Ro = {(x, y) | x # YI. < 


There is another way that relations are combined that is analogous to the composition of 
functions. 
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DEFINITION 6 


EXAMPLE 20 


EXAMPLE 21 


DEFINITION 7 


EXAMPLE 22 


THEOREM 1 


Let R be arelation from a set A to a set B and S a relation from B to a set C. The composite 
of R and S is the relation consisting of ordered pairs (a,c), where a € A, c € C, and for 
which there exists an element b € B such that (a,b) € R and (b,c) € S. We denote the 
composite of R and S by SOR. 


Computing the composite of two relations requires that we find elements that are the second 
element of ordered pairs in the first relation and the first element of ordered pairs in the second 
relation, as Examples 20 and 21 illustrate. 


What is the composite of the relations R and S, where R is the relation from {1, 2, 3} to {1, 2, 3, 4} 
with R = {(1, 1), C1, 4), (2, 3), (3, 1), (3, 4)} and S is the relation from {1, 2, 3, 4} to {0, 1, 2} 
with S = {(1, 0), (2, 0), (3, 1), G, 2), 4, D}? 


Solution: § © R is constructed using all ordered pairs in R and ordered pairs in S, where the 
second element of the ordered pair in R agrees with the first element of the ordered pair 
in S. For example, the ordered pairs (2, 3) in R and (3, 1) in S produce the ordered pair (2, 1) 
in S oR. Computing all the ordered pairs in the composite, we find 


SoR={(,0), , 1), 2, 1), @, 2), B, 0), 3, D}. < 


Composing the Parent Relation with Itself Let R be the relation on the set of all people 
such that (a, b) € R if person a is a parent of person b. Then (a, c) € ROR if and only if there 
is a person b such that (a, b) € R and (b,c) € R, that is, if and only if there is a person b such 
that a is a parent of b and b is a parent of c. In other words, (a,c) € ROR if and only if aisa 
grandparent of c. < 


The powers of a relation R can be recursively defined from the definition of a composite of 
two relations. 


Let R be a relation on the set A. The powers R”,n = 1, 2,3,..., are defined recursively by 


R'=R and R'+1— RoR, 


The definition shows that R? = RoR, R? = R20R =(ROR)OR, and 0 on. 
Let R = {(1, 1), (2, 1), G, 2), (4, 3)}. Find the powers R",n = 2,3,4,.... 


Solution: Because R* = ROR, we find that R? = {(1, 1), (2, 1), (3, 1), (4, 2)}. Further- 
more, because R? = R20 R, R? = {(1, 1), (2, 1), (3, 1), (4, 1)}. Additional computation shows 
that R* is the same as R?, so R* = {(1, 1), (2, 1), (3, 1), (4, I}. It also follows that R” = R? 
forn = 5,6,7,.... The reader should verify this. < 


The following theorem shows that the powers of a transitive relation are subsets of this 
relation. It will be used in Section 9.4. 


The relation R on a set A is transitive if and only if R” C R forn = 1,2,3,.... 


1. List the ordered pairs 
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Proof: We first prove the “if” part of the theorem. We suppose that R’ C R for n= 1, 
2,3,.... In particular, R2 C R. To see that this implies R is transitive, note that if (a,b) € R 
and (b, c) € R, then by the definition of composition, (a, c) € R?. Because R2 C R, this means 


that (a,c) € R. Hence, R is transitive. 


® We will use mathematical induction to prove the only if part of the theorem. Note that this 
part of the theorem is trivially true for n = 1. 

Assume that R” C R, where n is a positive integer. This is the inductive hypothesis. To 

complete the inductive step we must show that this implies that R’*! is also a subset of R. 

To show this, assume that (a,b) € R"+!. Then, because R”+! = R"OR, there is an 

element x with x € A such that (a, x) € R and (x, b) € R". The inductive hypothesis, namely, 

that R” C R, implies that (x, b) € R. Furthermore, because R is transitive, and (a,x) € R 

and (x, b) € R, it follows that (a, b) € R. This shows that R™*1 CR, completing the proof. <1 


Exercises 


in the relation R from 
A = {0, 1,2, 3,4} to B = {0, 1, 2,3}, where (a,b) ER 
if and only if 


a) a=b. b) a+b=4. 
c) a>b. d) a|b. 
e) gced(a, b) = 1. f) Icm(a, b) = 2. 
. a) List all the ordered pairs in the relation 


R = {(a, b) | a divides b} on the set {1, 2, 3, 4, 5, 6}. 

b) Display this relation graphically, as was done in 
Example 4. 

c) Display this relation in tabular form, as was done in 
Example 4. 

. For each of these relations on the set {1, 2, 3, 4}, decide 

whether it is reflexive, whether it is symmetric, whether 

it is antisymmetric, and whether it is transitive. 

a) {(2,2), (2,3), (2,4), 3,2), G,3), 3,4} 

b) {(1, 1), (1, 2), 2, 1), 2,2), 3,3), (4, 4)} 

e) (2,4), (4,2)} 

d) {(1,2), 2,3), 3,4} 

e) {(1, 1), 2,2), G,3), (4,4)} 

f) {0,3), 0,4), 2,3), 2,4), G,), G4} 

. Determine whether the relation R on the set of all people 

is reflexive, symmetric, antisymmetric, and/or transitive, 

where (a, b) € R if and only if 

a) ais taller than b. 

b) a and b were born on the same day. 

c) ahas the same first name as b. 

d) a and b have a common grandparent. 

. Determine whether the relation R on the set of all Web 

pages is reflexive, symmetric, antisymmetric, and/or tran- 

sitive, where (a, b) € R if and only if 

a) everyone who has visited Web page a has also visited 
Web page b. 

b) there are no common links found on both Web 
page a and Web page b. 

c) there is at least one common link on Web page a and 
Web page b. 


10. 


d) there is a Web page that includes links to both Web 
page a and Web page b. 


. Determine whether the relation R on the set of all real 


numbers is reflexive, symmetric, antisymmetric, and/or 
transitive, where (x, y) € R if and only if 


a) x+y=0. b) x =+y. 
c) x — y is arational number. 

d) x =2y. e) xy > 0. 
f) xy =0. g)x=1. 


h) x=lory=1. 


. Determine whether the relation R on the set of all integers 


is reflexive, symmetric, antisymmetric, and/or transitive, 
where (x, y) € R if and only if 

a) xy. b) xy > 1. 

ec) x=yt+lorx=y-1. 

d) x = y (mod 7). e) x isa multiple of y. 

f) x and y are both negative or both nonnegative. 

g) x=y*. hy ey. 


. Show that the relation R = 4 on anonempty set S is sym- 


metric and transitive, but not reflexive. 


. Show that the relation R = Y on the empty set S = 9 is 


reflexive, symmetric, and transitive. 
Give an example of a relation on a set that is 


a) both symmetric and antisymmetric. 
b) neither symmetric nor antisymmetric. 


A relation R on the set A is irreflexive if for every 
aéA,(a,a) ¢ R. That is, R is irreflexive if no element 
in A is related to itself. 


11. 
12. 
13. 
14. 
15. 
16. 


17. 


Which relations in Exercise 3 are irreflexive? 

Which relations in Exercise 4 are irreflexive? 

Which relations in Exercise 5 are irreflexive? 

Which relations in Exercise 6 are irreflexive? 

Can a relation on a set be neither reflexive nor irreflexive? 
Use quantifiers to express what it means for a relation to 
be irreflexive. 

Give an example of an irreflexive relation on the set of all 
people. 
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A relation R is called asymmetric if (a,b) € R implies that 
(b, a) ¢ R. Exercises 18-24 explore the notion of an asym- 
metric relation. Exercise 22 focuses on the difference between 
asymmetry and antisymmetry. 


18. 
19. 
20. 
21. 
22. 


23. 


24, 


25. 


Which relations in Exercise 3 are asymmetric? 
Which relations in Exercise 4 are asymmetric? 
Which relations in Exercise 5 are asymmetric? 
Which relations in Exercise 6 are asymmetric? 


Must an asymmetric relation also be antisymmetric? Must 
an antisymmetric relation be asymmetric? Give reasons 
for your answers. 


Use quantifiers to express what it means for a relation to 
be asymmetric. 


Give an example of an asymmetric relation on the set of 
all people. 


How many different relations are there from a set with m 
elements to a set with n elements? 


C= Let R be a relation from a set A to a set B. The inverse rela- 
tion from B to A, denoted by R—!, is the set of ordered pairs 
{(b, a) | (a,b) € R}. The complementary relation R is the 
set of ordered pairs {(a, b) | (a, b) € R}. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


Let R be the relation R = {(a, b) | a < b} on the set of 
integers. Find 

a) Ro, b) R. 

Let R be the relation R = {(a, b) | a divides b} on the set 
of positive integers. Find 

a) Ro. b) R. 

Let R be the relation on the set of all states in the United 
States consisting of pairs (a, b) where state a borders 
state b. Find 

a) Ro. b) R. 

Suppose that the function f from A to B is a one-to- 
one correspondence. Let R be the relation that equals the 
graph of f. Thatis, R = {(a, f(a)) | a € A}. What is the 
inverse relation R~!? 


Let R; = {(1, 2), (2, 3), (3, 4)} and Ro = {(1, 1), C1, 2), 
(2, 1), (2, 2), (2, 3), 3, 1), G, 2), 3, 3), GB, 4)} be rela- 
tions from {1, 2, 3} to {1, 2, 3, 4}. Find 

a) Ri UR. b) RYN Ro. 

ec) Rj — Ro. d) Ro— Rj. 

Let A be the set of students at your school and B the set of 
books in the school library. Let Rj and R>2 be the relations 
consisting of all ordered pairs (a, b), where student a is 
required to read book b in a course, and where student a 
has read book b, respectively. Describe the ordered pairs 
in each of these relations. 


a) Rj UR> b) RYN Ro 

c) Ri @ Ro d) Ri — Ro 

e) Ry— Ri 

Let R be the relation {(1, 2), (1, 3), (2, 3), (2,4), G, D}, 


and let S be the relation {(2, 1), (3, 1), @G, 2), (4, 2)}. 
Find SoR. 


33. 


Let R be the relation on the set of people consisting of 
pairs (a, b), where a is a parent of b. Let S be the relation 
on the set of people consisting of pairs (a, b), where a 
and b are siblings (brothers or sisters). What are § 0 R 
and Ro S? 


Exercises 34-37 deal with these relations on the set of real 
numbers: 


R, ={(a,by€ R2 | a > b}, the “greater than” relation, 
Ro = {(a, by € R2 | a => b}, the “greater than or equal to” 


relation, 


R3 = {(a, b) € R? | a < 5}, the “less than” relation, 
R4 = {(a, b) € R? | a < b}, the “less than or equal to” 


relation, 


Rs = {(a, b) € R? | a = D}, the “equal to” relation, 
Ro = {(a,b) € R? | a # b}, the “unequal to” relation. 


34. 


35. 


36. 


37. 


38. 


39. 


40. 


Find 

a) R, UR3. b) RI URs. 
ce) Ro Ry. d) R30 Rs. 
e) Rj — Ro. f) Ro — Rj. 
g) Ri GR. h) Ro @ R4. 
Find 

a) RoU R4. b) R3U Ro. 
c) R3M Ro. d) R4N Ro. 
e) R3— Ro. f) Ro — R3. 
g) Ro ® Ro. h) R3 @ Rs. 
Find 

a) R,o Rt. b) Rio Ro. 
c) Rio R3. d) Rio Ra. 
e) Rio Rs. f) Rio Ro. 
g) Roo R3. h) R30 R3. 
Find 

a) Roo Rj. b) Roo Ro. 
c) R30 Rs. d) Rago R}. 
e) R50 R3. f) R30 Ro. 
g) Ryo Ro. h) Roo Ro. 
Let R be the parent relation on the set of all people (see 


Example 21). When is an ordered pair in the relation R3? 


Let R be the relation on the set of people with doctorates 
such that (a, b) € Rif and only if a was the thesis advisor 
of b. When is an ordered pair (a, b) in R2? When is an 
ordered pair (a, b) in R", when n is a positive integer? 
(Assume that every person with a doctorate has a thesis 
advisor.) 


Let R; and Rp» be the “divides” and “is a multiple of” 
relations on the set of all positive integers, respectively. 
That is, Ry = {(a, b) | a divides b} and R2 = {(a, b) | a 
is a multiple of b}. Find 


a) R, UR>. b) Rp Ro. 
c) Ry — Ro. d) Ro— Rj. 
e) Ri @ Ro. 


41. 


42. 
43. 


44. 


45. 


46. 


*47, 


* 48. 


Let R; and R> be the “congruent modulo 3” and the 
“congruent modulo 4” relations, respectively, on the set 
of integers. That is, R; = {(a, b) | a = b (mod 3)} and 
Ro = {(a, b) | a = b (mod 4)}. Find 


a) R, UR>. b) RYN Ro. 
c) Ry — Ro. d) Ro — Rj. 
e) R; ® Ro. 


List the 16 different relations on the set {0, 1}. 

How many of the 16 different relations on {0, 1} contain 
the pair (0, 1)? 

Which of the 16 relations on {0, 1}, which you listed in 
Exercise 42, are 
a) reflexive? 

c) symmetric? 
e) asymmetric? 


b) irreflexive? 

d) antisymmetric? 

f) transitive? 

a) How many relations are there on the set {a, b, c, d}? 

b) How many relations are there on the set {a, b, c, d} 
that contain the pair (a, a)? 

Let S be a set with n elements and let a and b be dis- 

tinct elements of S. How many relations R are there on S$ 

such that 

a) (a,b) € R? b) (a,b) g R? 

c) no ordered pair in R has a as its first element? 

d) at least one ordered pair in R has a as its first element? 

e) no ordered pair in R has a as its first element or b as 
its second element? 

f) at least one ordered pair in R either has a as its first 
element or has b as its second element? 


How many relations are there on a set with n elements 
that are 

a) symmetric? 

c) asymmetric? 

e) reflexive and symmetric? 
f) neither reflexive nor irreflexive? 


b) antisymmetric? 
d) irreflexive? 


How many transitive relations are there on a set with n 
elements if 
a) n=1? 


b) n=2? c) n=3? 


49. 


50. 


51. 


52. 


53. 


54. 


55: 


56. 


57. 


* 58. 


59. 
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Find the error in the “proof” of the following “theorem.” 


“Theorem”: Let R be a relation on a set A that is sym- 
metric and transitive. Then R is reflexive. 


“Proof”: Let a € A. Take an element b € A such that 
(a,b) € R. Because R is symmetric, we also have 
(b, a) € R. Now using the transitive property, we can con- 
clude that (a, a) € R because (a, b) € Rand (b, a) € R. 
Suppose that R and S are reflexive relations on a set A. 
Prove or disprove each of these statements. 

a) RUS is reflexive. 

b) ROS is reflexive. 

c) R@ S is irreflexive. 

d) R — Sis irreflexive. 

e) SOR is reflexive. 

Show that the relation R on a set A is symmetric if and 
only if R = R—!, where R7! is the inverse relation. 
Show that the relation R on a set A is antisymmetric if 
and only if R  R7! isa subset of the diagonal relation 
A = {(a,a) | a € A}. 

Show that the relation R on a set A is reflexive if and only 
if the inverse relation R~! is reflexive. 

Show that the relation R on a set A is reflexive if and only 
if the complementary relation R is irreflexive. 

Let R be a relation that is reflexive and transitive. Prove 
that R” = R for all positive integers n. 

Let R be the relation on the set {1, 2, 3, 4, 5} containing 
the ordered pairs (1, 1), (1, 2), (1, 3), (2, 3), (2, 4), (3, 1), 
(3,4), (3,5), (4,2), (4,5), (5, 1D, (6,2), and (5, 4). 
Find 

a) R?. b) R°. c) Rt. d) R>. 

Let R be a reflexive relation on a set A. Show that R” is 
reflexive for all positive integers n. 

Let R be asymmetric relation. Show that R” is symmetric 
for all positive integers n. 

Suppose that the relation R is irreflexive. Is R? necessar- 
ily irreflexive? Give a reason for your answer. 


n-ary Relations and Their Applications 


Introduction 


Relationships among elements of more than two sets often arise. For instance, there is a relation- 
ship involving the name of a student, the student’s major, and the student’s grade point average. 
Similarly, there is a relationship involving the airline, flight number, starting point, destination, 
departure time, and arrival time of a flight. An example of such a relationship in mathematics 
involves three integers, where the first integer is larger than the second integer, which is larger 
than the third. Another example is the betweenness relationship involving points on a line, such 
that three points are related when the second point is between the first and the third. 

We will study relationships among elements from more than two sets in this section. These re- 
lationships are called n-ary relations. These relations are used to represent computer databases. 
These representations help us answer queries about the information stored in databases, such 
as: Which flights land at O’Hare Airport between 3 A.M. and 4 A.M.? Which students at your 
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DEFINITION 1 


EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


Links 


school are sophomores majoring in mathematics or computer science and have greater than a 
3.0 average? Which employees of a company have worked for the company less than 5 years 
and make more than $50,000? 


n-ary Relations 


We begin with the basic definition on which the theory of relational databases rests. 


Let A, A2,..., Ay be sets. Ann-ary relation on these sets isa subset of A; x Az x --- x An. 
The sets Aj, Az,..., A, are called the domains of the relation, and n is called its degree. 


Let R be the relation on N x N x N consisting of triples (a, b, c), where a, b, and c are integers 
witha < b < c.Then(1, 2,3) € R, but (2, 4, 3) ¢ R. The degree of this relation is 3. Its domains 
are all equal to the set of natural numbers. < 


Let R be the relation on Z x Z x Z consisting of all triples of integers (a,b,c) in which 
a, b, and c form an arithmetic progression. That is, (a,b,c) € R if and only if there is 
an integer k such thatb=a-+k and c=a+ 2k, or equivalently, such that b-—a =k and 
c—b=k. Note that (1, 3,5) € R because 3 = 1+2 and5=1+2- 2, but (2,5,9) ¢ R be- 
cause 5 — 2 = 3 while 9 — 5 = 4. This relation has degree 3 and its domains are all equal to the 
set of integers. < 


Let R be the relationonZ x Z x Z* consisting of triples (a, b, m), where a, b, and m are integers 
with m > | and a = b (modm). Then (8, 2, 3), (—1, 9, 5), and (14, 0, 7) all belong to R, but 
(7, 2, 3), (—2, —8, 5), and (11, 0, 6) donot belong to R because 8 = 2 (mod 3), —1 = 9 (mod 5), 
and 14 = 0 (mod 7), but 7 ¥ 2 (mod 3), —2 4 —8 (mod 5), and 11 # 0 (mod 6). This relation 
has degree 3 and its first two domains are the set of all integers and its third domain is the set of 
positive integers. < 


Let R be the relation consisting of 5-tuples (A, VN, S, D, T) representing airplane flights, 
where A is the airline, N is the flight number, S is the starting point, D is the destination, and T is 
the departure time. For instance, if Nadir Express Airlines has flight 963 from Newark to Bangor 
at 15:00, then (Nadir, 963, Newark, Bangor, 15:00) belongs to R. The degree of this relation 
is 5, and its domains are the set of all airlines, the set of flight numbers, the set of cities, the set 
of cities (again), and the set of times. < 


Databases and Relations 


The time required to manipulate information in a database depends on how this information is 
stored. The operations of adding and deleting records, updating records, searching for records, 
and combining records from overlapping databases are performed millions of times each day ina 
large database. Because of the importance of these operations, various methods for representing 
databases have been developed. We will discuss one of these methods, called the relational 
data model, based on the concept of a relation. 

A database consists of records, which are n-tuples, made up of fields. The fields are the 
entries of the n-tuples. For instance, a database of student records may be made up of fields 
containing the name, student number, major, and grade point average of the student. The rela- 
tional data model represents a database of records as an n-ary relation. Thus, student records 
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TABLE 1 Students. 


Ackermann 231455 Computer Science 
Adams 888323 Physics 


Chou 102147 Computer Science 
Goodfriend 453876 Mathematics 

Rao 678543 Mathematics 
Stevens 786576 Psychology 


are represented as 4-tuples of the form (Student_name, ID_number, Major, GPA). A sample 
database of six such records is 


(Ackermann, 231455, Computer Science, 3.88) 
(Adams, 888323, Physics, 3.45) 

(Chou, 102147, Computer Science, 3.49) 
(Goodfriend, 453876, Mathematics, 3.45) 
(Rao, 678543, Mathematics, 3.90) 

(Stevens, 786576, Psychology, 2.99). 


Relations used to represent databases are also called tables, because these relations are often 
displayed as tables. Each column of the table corresponds to an attribute of the database. For 
instance, the same database of students is displayed in Table 1. The attributes of this database 
are Student Name, ID Number, Major, and GPA. 

A domain of an n-ary relation is called a primary key when the value of the n-tuple from 
this domain determines the n-tuple. That is, a domain is a primary key when no two n-tuples in 
the relation have the same value from this domain. 

Records are often added to or deleted from databases. Because of this, the property that a 
domain is a primary key is time-dependent. Consequently, a primary key should be chosen that 
remains one whenever the database is changed. The current collection of n-tuples in a relation 
is called the extension of the relation. The more permanent part of a database, including the 
name and attributes of the database, is called its intension. When selecting a primary key, the 
goal should be to select a key that can serve as a primary key for all possible extensions of the 
database. To do this, it is necessary to examine the intension of the database to understand the 
set of possible n-tuples that can occur in an extension. 


Which domains are primary keys for the n-ary relation displayed in Table 1, assuming that no 
n-tuples will be added in the future? 


Solution: Because there is only one 4-tuple in this table for each student name, the domain 
of student names is a primary key. Similarly, the ID numbers in this table are unique, so the 
domain of ID numbers is also a primary key. However, the domain of major fields of study 
is not a primary key, because more than one 4-tuple contains the same major field of study. 
The domain of grade point averages is also not a primary key, because there are two 4-tuples 
containing the same GPA. < 


Combinations of domains can also uniquely identify n-tuples in an n-ary relation. When 
the values of a set of domains determine an n-tuple in a relation, the Cartesian product of these 
domains is called a composite key. 
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Is the Cartesian product of the domain of major fields of study and the domain of GPAs a 
composite key for the n-ary relation from Table |, assuming that no n-tuples are ever added? 


Solution: Because no two 4-tuples from this table have both the same major and the same GPA, 
this Cartesian product is a composite key. 


Because primary and composite keys are used to identify records uniquely in a database, 
it is important that keys remain valid when new records are added to the database. Hence, 
checks should be made to ensure that every new record has values that are different in 
the appropriate field, or fields, from all other records in this table. For instance, it makes 
sense to use the student identification number as a key for student records if no two 
students ever have the same student identification number. A university should not use the 
name field as a key, because two students may have the same name (such as John Smith). 


Operations on n-ary Relations 


There are a variety of operations on n-ary relations that can be used to form new n-ary relations. 
Applied together, these operations can answer queries on databases that ask for all n-tuples that 
satisfy certain conditions. 

The most basic operation on an n-ary relation is determining all n-tuples in the n-ary relation 
that satisfy certain conditions. For example, we may want to find all the records of all computer 
science majors in a database of student records. We may want to find all students who have a 
grade point average above 3.5. We may want to find the records of all computer science majors 
who have a grade point average above 3.5. To perform such tasks we use the selection operator. 


Let R be ann-ary relation and C a condition that elements in R may satisfy. Then the selection 
operator sc maps the n-ary relation R to the n-ary relation of all n-tuples from R that satisfy 
the condition C. 


To find the records of computer science majors in the n-ary relation R shown in Table 1, we use 
the operator sc,, where C; is the condition Major = “Computer Science.” The result is the two 
4-tuples (Ackermann, 231455, Computer Science, 3.88) and (Chou, 102147, Computer Science, 
3.49). Similarly, to find the records of students who have a grade point average above 3.5 in 
this database, we use the operator sc,, where C2 is the condition GPA > 3.5. The result is the 
two 4-tuples (Ackermann, 231455, Computer Science, 3.88) and (Rao, 678543, Mathematics, 
3.90). Finally, to find the records of computer science majors who have a GPA above 3.5, we 
use the operator sc,, where C3 is the condition (Major = “Computer Science” A GPA > 3.5). 
The result consists of the single 4-tuple (Ackermann, 231455, Computer Science, 3.88). < 


Projections are used to form new n-ary relations by deleting the same fields in every record 


of the relation. 


The projection P;,i,,...,i, where ij < i2 <--+- < im, maps the n-tuple (a1, a2, ..., dy) to the 
m-tuple (dj,, aj,,..., Gi,,), where m <n. 


In other words, the projection Pj, i5,...,i,, 
the 7; th, ith, ..., and i,,th components. 


deletes n — m of the components of an n-tuple, leaving 


TABLE 2 GPAs. 


Student_name GPA 


Ackermann 
Adams 
Chou 
Goodfriend 
Rao 


Stevens 


EXAMPLE 8 


EXAMPLE 9 


EXAMPLE 10 


DEFINITION 4 
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TABLE 3 Enrollments. TABLE 4 Majors. 


Glauser Biology Glauser Biology 
Glauser Biology Marcus Mathematics 


Glauser Biology Miller Computer Science 


Marcus Mathematics 
Marcus Mathematics 
Marcus Mathematics 
Miller Computer Science 
Miller Computer Science 


What results when the projection P)3 is applied to the 4-tuples (2,3,0,4), 
(Jane Doe, 234111001, Geography, 3.14), and (a1, a2, a3, a4)? 


Solution: The projection P; 3 sends these 4-tuples to (2, 0), (Jane Doe, Geography), and (a1, a3), 
respectively. < 


Example 9 illustrates how new relations are produced using projections. 


What relation results when the projection P} 4 is applied to the relation in Table 1? 


Solution: When the projection P 4 is used, the second and third columns of the table are deleted, 
and pairs representing student names and grade point averages are obtained. Table 2 displays 
the results of this projection. < 


Fewer rows may result when a projection is applied to the table for a relation. This happens 
when some of the n-tuples in the relation have identical values in each of the m components of 
the projection, and only disagree in components deleted by the projection. For instance, consider 
the following example. 


What is the table obtained when the projection P\,2 is applied to the relation in Table 3? 


Solution: Table 4 displays the relation obtained when P;,2 is applied to Table 3. Note that there 
are fewer rows after this projection is applied. < 


The join operation is used to combine two tables into one when these tables share some 
identical fields. For instance, a table containing fields for airline, flight number, and gate, and 
another table containing fields for flight number, gate, and departure time can be combined into 
a table containing fields for airline, flight number, gate, and departure time. 


Let R be a relation of degree m and S a relation of degree n. The join Jp(R, S), 
where p<m and p<vn, is a relation of degree m+ n-— pp that consists of all 
(m +n — p)-tuples (a1, a2, ...,; dm—ps C1, C2, +++, Cp, D1, b2,..., bn—p), where the m-tuple 
(a1, 42, ...,Am—p,C1,€2,..., Cp) belongs to R and the n-tuple (c1, c2,..., Cp, b1, b2,..., 
bn—p) belongs to S. 


In other words, the join operator J, produces a new relation from two relations by combining all 
m-tuples of the first relation with all n-tuples of the second relation, where the last p components 
of the m-tuples agree with the first p components of the n-tuples. 
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TABLE 5 Teaching_assignments. TABLE 6 Class_schedule. 


Course_ Course_ 
Professor Department number Department number Time 


Cruz Zoology Computer Science 


Cruz Zoology Mathematics 


Farber Psychology Mathematics 


Farber Psychology Physics 


Grammer Physics Psychology 


Grammer Physics Psychology 


Rosen Computer Science Zoology 


Rosen Mathematics Zoology 


EXAMPLE 11 


Links 


EXAMPLE 12 


What relation results when the join operator J2 is used to combine the relation displayed in 
Tables 5 and 6? 


Solution: The join Jz produces the relation shown in Table 7. < 


There are other operators besides projections and joins that produce new relations from 
existing relations. A description of these operations can be found in books on database theory. 


SOL 


The database query language SQL (short for Structured Query Language) can be used to carry 
out the operations we have described in this section. Example 12 illustrates how SQL commands 
are related to operations on n-ary relations. 


We will illustrate how SQL is used to express queries by showing how SQL can be employed 
to make a query about airline flights using Table 8. The SQL statement 


SELECT Departure_time 
FROM Flights 
WHERE Destination=‘Detroit’ 


is used to find the projection Ps (on the Departure_time attribute) of the selection of 5-tuples in 
the Flights database that satisfy the condition: Destination = ‘Detroit’. The output would be a 
list containing the times of flights that have Detroit as their destination, namely, 08:10, 08:47, 


TABLE 7 Teaching_schedule. 


rotor | Demet | Cnr mtr tao | Tie | 
335 


Cruz Zoology 


Cruz Zoology 412 


Farber Psychology 501 


Farber Psychology 617 
Grammer Physics 544 
Rosen Computer Science 518 
Rosen Mathematics 575 
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TABLE 8 Flights. 


122 34 


Detroit 
Denver 


Anchorage 


Honolulu 
Detroit 
Denver 
Detroit 


and 09:44. SQL uses the FROM clause to identify the n-ary relation the query is applied to, the 
WHERE clause to specify the condition of the selection operation, and the SELECT clause to 
specify the projection operation that is to be applied. (Beware: SQL uses SELECT to represent 
a projection, rather than a selection operation. This is an unfortunate example of conflicting 
terminology.) < 


Example 13 shows how SQL queries can be made involving more than one table. 


is used to find the projection P},5 of the 5-tuples in the database (shown in Table 7), which 
is the join Jz of the Teaching_assignments and Class_schedule databases in Tables 5 and 6, 
respectively, which satisfy the condition: Department = Mathematics. The output would consist 
of the single 2-tuple (Rosen, 3:00 p.M.). The SQL FROM clause is used here to find the join of 
two different databases. < 


We have only touched on the basic concepts of relational databases in this section. More 


EXAMPLE 13 The SQL statement 
SELECT Professor, Time 
FROM Teaching_assignments, Class_schedule 
WHERE Department=’Mathematics’ 
information can be found in [AhUI195]. 
Exercises 


1. List the triples in the relation {(a, b, c) | a, b, and c are 
integers withO <a <b <c <5}. 

2. Which 4-tuples are in the relation {(a, b, c,d) | a, b, c, 
and d are positive integers with abcd = 6}? 

3. List the 5-tuples in the relation in Table 8. 

4. Assuming that no new n-tuples are added, find all the 
primary keys for the relations displayed in 


a) Table 3. 
c) Table 6. 


7. The 3-tuples in a 3-ary relation represent the following 
attributes of a student database: student ID number, name, 
phone number. 

a) Is student ID number likely to be a primary key? 
b) Is name likely to be a primary key? 
c) Is phone number likely to be a primary key? 


8. The 4-tuples in a 4-ary relation represent these attributes 
of published books: title, ISBN, publication date, number 
of pages. 


b) Table 5. 
d) Table 8. 


5. Assuming that no new n-tuples are added, find a compos- 


ite key with two fields containing the Airline field for the 
database in Table 8. 

. Assuming that no new n-tuples are added, find a compos- 
ite key with two fields containing the Professor field for 
the database in Table 7. 


a) What is a likely primary key for this relation? 

b) Under what conditions would (title, publication date) 
be a composite key? 

c) Under what conditions would (title, number of pages) 
be a composite key? 
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9. 


10. 
11. 
12. 
13. 


14. 
15. 
16. 
17. 


18. 


19. 


20. 
21. 


22. 


TABLE 9 Part_needs. 


Supplier 


The 5-tuples in a 5-ary relation represent these attributes 

of all people in the United States: name, Social Security 

number, street address, city, state. 

a) Determine a primary key for this relation. 

b) Under what conditions would (name, street address) 
be a composite key? 

c) Under what conditions would (name, street address, 
city) be a composite key? 


What do you obtain when you apply the selection oper- 
ator sc, where C is the condition Room = A100, to the 
database in Table 7? 

What do you obtain when you apply the selection oper- 
ator sc, where C is the condition Destination = Detroit, 
to the database in Table 8? 

What do you obtain when you apply the selection op- 
erator sc, where C is the condition (Project =2) A 
(Quantity > 50), to the database in Table 10? 

What do you obtain when you apply the selection oper- 
ator sc, where C is the condition (Airline = Nadir) v 
(Destination = Denver), to the database in Table 8? 
What do you obtain when you apply the projection P2,3\5 
to the 5-tuple (a, b, c, d, e)? 

Which projection mapping is used to delete the first, 
second, and fourth components of a 6-tuple? 

Display the table produced by applying the projection 
P,2,4 to Table 8. 

Display the table produced by applying the projection 
P,,4 to Table 8. 

How many components are there in the n-tuples in the 
table obtained by applying the join operator J3 to two 
tables with 5-tuples and 8-tuples, respectively? 
Construct the table obtained by applying the join 
operator J to the relations in Tables 9 and 10. 

Show that if C; and Cy) are conditions that el- 
ements of the n-ary relation R may satisfy, then 
SC, ACy (R) = Sc, (Scy (R)). 

Show that if Cy; and Cp» are conditions that ele- 
ments of the n-ary relation R may satisfy, then 
SC, (SCy(R)) = Sc, (8c, (R)). 

Show that if C is a condition that elements of 
the n-ary relations R and S may satisfy, then 
sc(R U S) = sc(R) U sc(S). 


Part_number 


1 
3 
4 
3 
2 
4 
2 
1 


Pi 


23. 


24, 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


Show that if C is a condition that elements of 
the n-ary relations R and S may satisfy, then 
sc(RN S) = sc(R) ON sc (S). 


Show that if C is a condition that elements of 
the n-ary relations R and S may satisfy, then 
sc(R — S) = sc(R) — sc(S). 


Show that if R and S are both n-ary relations, then 
Piy in (RUS) = Pi in (R) U Piy,iz,.sim CS). 


Give an example to show that if R and S are both n-ary 
relations, then Pj, i,,...,i,,(R OS) may be different from 
Pit sin, .ceim CR) O Pit ,in,..-sim (S)- 


Give an example to show that if R and S are both n-ary 
relations, then Pj, j,,...,i,,(R — S) may be different from 
Pit ins sim OR) — Pit in, (S)- 


a) What are the operations that correspond to the query 
expressed using this SQL statement? 


pees Im sel 


SELECT Supplier 
FROM Part_needs 
WHERE 1000 < Part_number < 5000 


b) What is the output of this query given the database in 
Table 9 as input? 


a) What are the operations that correspond to the query 
expressed using this SQL statement? 


SELECT Supplier, Project 
FROM Part_needs, Parts_inventory 
WHERE Quantity < 10 


b) What is the output of this query given the databases 
in Tables 9 and 10 as input? 


Determine whether there is a primary key for the relation 
in Example 2. 


Determine whether there is a primary key for the relation 
in Example 3. 


Show that an n-ary relation with a primary key can be 
thought of as the graph of a function that maps values of 
the primary key to (n — 1)-tuples formed from values of 
the other domains. 


TABLE 10 Parts_inventory. 


NF RPRWN WR 
BNF NN YF LY COO 
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Representing Relations 


EXAMPLE 1 


EXAMPLE 2 


Introduction 


In this section, and in the remainder of this chapter, all relations we study will be binary relations. 
Because of this, in this section and in the rest of this chapter, the word relation will always refer 
to a binary relation. There are many ways to represent a relation between finite sets. As we have 
seen in Section 9.1, one way is to list its ordered pairs. Another way to represent a relation is to 
use a table, as we did in Example 3 in Section 9.1. In this section we will discuss two alternative 
methods for representing relations. One method uses zero—one matrices. The other method uses 
pictorial representations called directed graphs, which we will discuss later in this section. 

Generally, matrices are appropriate for the representation of relations in computer programs. 
On the other hand, people often find the representation of relations using directed graphs useful 
for understanding the properties of these relations. 


Representing Relations Using Matrices 


A relation between finite sets can be represented using a zero—one matrix. Suppose that R is a 
relation from A = {a),d2,...,@m} to B = {bj, bo, ..., b,}. (Here the elements of the sets A 
and B have been listed in a particular, but arbitrary, order. Furthermore, when A = B we use 
the same ordering for A and B.) The relation R can be represented by the matrix Mr = [mij], 
where 


1 if (qj, bj) = R, 
Mij=),. 
0 if (di, bj) ¢ R. 
In other words, the zero—one matrix representing R has a | as its (i, 7) entry when q; is related 
to b;, and a 0 in this position if a; is not related to b;. (Such a representation depends on the 


orderings used for A and B.) 
The use of matrices to represent relations is illustrated in Examples 1-6. 


Suppose that A = {1, 2, 3} and B = {1, 2}. Let R be the relation from A to B containing (a, b) 
ifa € A,b € B, anda > b. What is the matrix representing R if a; = 1, az = 2, and a3 = 3, 
and bj = 1 and bo = 2? 


Solution: Because R = {(2, 1), (3, 1), (3, 2)}, the matrix for R is 
0 0 
Mr=|1 0 
1 1 


The Is in Mp show that the pairs (2, 1), (3, 1), and (3, 2) belong to R. The Os show that no 
other pairs belong to R. < 


Let A = {a1, a2, a3} and B = {b, b2, b3, b4, bs}. Which ordered pairs are in the relation R 
represented by the matrix 


01000 
Mre=|1 011 0? 
10101 
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FIGURE1 The 
Zero—One Matrix 
for a Reflexive 
Relation. (Off 
Diagonal Ele- 
ments Can 

Be 0 or 1.) 


EXAMPLE 3 


Solution: Because R consists of those ordered pairs (a;, b;) with m;; = 1, it follows that 
R = {(a1, 52), (a2, b1), (a2, 53), (a2, b4), (a3, b1), (a3, 63), (a3, Ds)}. < 


The matrix of a relation on a set, which is a square matrix, can be used to determine whether 
the relation has certain properties. Recall that arelation R on A is reflexive if (a, a) € R whenever 
a € A. Thus, R is reflexive if and only if (a;, a;) € R fori = 1,2,...,n. Hence, R is reflexive 
if and only if m;; = 1, fori = 1, 2,...,n. In other words, R is reflexive if all the elements on 
the main diagonal of Mp are equal to 1, as shown in Figure |. Note that the elements off the 
main diagonal can be either 0 or 1. 

The relation R is symmetric if (a,b) € R implies that (b,a) € R. Consequently, the 
relation R on the set A = {a), a2,..., dy} is symmetric if and only if (a;, a;) € R whenever 
(aj, a;) € R. In terms of the entries of Mr, R is symmetric if and only if mj; = 1 whenever 
mij = 1. This also means mj; = 0 whenever m;; = 0. Consequently, R is symmetric if and 
only if mj; = mj;, for all pairs of integers i and j withi = 1,2,...,n and j = 1,2,...,n. 
Recalling the definition of the transpose of a matrix from Section 2.6, we see that R is symmetric 
if and only if 


Mr = (Mpr)’, 


that is, if Mr is asymmetric matrix. The form of the matrix for a symmetric relation is illustrated 
in Figure 2(a). 

The relation R is antisymmetric if and only if (a,b) € R and (b, a) € R imply that a = b. 
Consequently, the matrix of an antisymmetric relation has the property that if m;; = 1 with 
i # j, then mj; = O. Or, in other words, either mj; = 0 or mj; = 0 wheni ¥ j. The form of 
the matrix for an antisymmetric relation is illustrated in Figure 2(b). 


(a) Symmetric (b) Antisymmetric 


FIGURE 2 The Zero—One Matrices for 
Symmetric and Antisymmetric Relations. 


Suppose that the relation R on a set is represented by the matrix 


1 0 
Mr=|1 1 1 
011 


Is R reflexive, symmetric, and/or antisymmetric? 


Solution: Because all the diagonal elements of this matrix are equal to 1, R is reflexive. Moreover, 
because Mr is symmetric, it follows that R is symmetric. It is also easy to see that R is not 
antisymmetric. < 


The Boolean operations join and meet (discussed in Section 2.6) can be used to find the 
matrices representing the union and the intersection of two relations. Suppose that R; and Ro 
are relations on a set A represented by the matrices Mr, and Mp,, respectively. The matrix 


EXAMPLE 4 


EXAMPLE 5 
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representing the union of these relations has a | in the positions where either Mr, or Mp, has 
a 1. The matrix representing the intersection of these relations has a | in the positions where 
both Mr, and Ma, have a 1. Thus, the matrices representing the union and intersection of these 
relations are 


Mrwrs = Mr, V Mr, and Mrinr> = Mr, A Mr,- 


Suppose that the relations R; and R2 on a set A are represented by the matrices 


101 1 0 
Mr, =/]1 0 0 and Mr, =|0 1 
0 1 0 1 0 


ore 


What are the matrices representing R; U R2 and R, NM R2? 


Solution: The matrices of these relations are 


1 01 
Mrur = Mr, Vv Mr, = 5 ; ’ 

1 0 1 
Mrink> = Ma, A Mr, = ; : 


< 


We now turn our attention to determining the matrix for the composite of relations. This 
matrix can be found using the Boolean product of the matrices (discussed in Section 2.6) for 
these relations. In particular, suppose that R is a relation from A to B and S is a relation 
from B to C. Suppose that A, B, and C have m, n, and p elements, respectively. Let the zero— 
one matrices for SOR, R, and S be Msor = [tj], Mr = [rij], and Ms = [s;;], respectively 
(these matrices have sizes m x p,m Xn, and n x p, respectively). The ordered pair (a;, c;) 
belongs to So R if and only if there is an element by such that (a;, b;) belongs to R and (bx, c;) 
belongs to S. It follows that t;; = 1 if and only if rj, = sx; = 1 for some k. From the definition 
of the Boolean product, this means that 


Ms.r = MrOMs. 
Find the matrix representing the relations S o R, where the matrices representing R and S are 


0 1 0 1 0 
Mr= 1 0 and Ms=/]0 0 1 
0 0 101 


orrF 


Solution: The matrix for SOR is 


1 11 
Ms.rk =MrROMs=]/0 1 1 
0 0 0 
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Q 
> 


d c 


FIGURE 3 
A Directed Graph. 


The matrix representing the composite of two relations can be used to find the matrix 
for Mrz. In particular, 


M R= Ml ‘ 
from the definition of Boolean powers. Exercise 35 asks for a proof of this formula. 


Find the matrix representing the relation R”, where the matrix representing R is 
0 1 0 
Mr=]0 1 1 
1 0 O 
Solution: The matrix for R? is 


Mp = MU! = 


oe ae) 
_— 


1 
1 
0 


Representing Relations Using Digraphs 


We have shown that a relation can be represented by listing all of its ordered pairs or by 
using a zero—one matrix. There is another important way of representing a relation using a 
pictorial representation. Each element of the set is represented by a point, and each ordered 
pair is represented using an arc with its direction indicated by an arrow. We use such pictorial 
representations when we think of relations on a finite set as directed graphs, or digraphs. 


A directed graph, or digraph, consists of a set V of vertices (or nodes) together with a set 
E of ordered pairs of elements of V called edges (or arcs). The vertex a is called the initial 
vertex of the edge (a, b), and the vertex b is called the terminal vertex of this edge. 


An edge of the form (a, a) is represented using an arc from the vertex a back to itself. Such 
an edge is called a loop. 


The directed graph with vertices a, b, c, and d, and edges (a, b), (a, d), (b, b), (b, d), (c, a 
(c, b), and (d, b) is displayed in Figure 3. 


The relation R ona set A is represented by the directed graph that has the elements of A as its 
vertices and the ordered pairs (a, b), where (a, b) € R, as edges. This assignment sets up a one- 
to-one correspondence between the relations on a set A and the directed graphs with A as their 
set of vertices. Thus, every statement about relations corresponds to a statement about directed 
graphs, and vice versa. Directed graphs give a visual display of information about relations. As 
such, they are often used to study relations and their properties. (Note that relations from a set 
A to a set B can be represented by a directed graph where there is a vertex for each element of 
A and a vertex for each element of B, as shown in Section 9.1. However, when A = B, such 
representation provides much less insight than the digraph representations described here.) The 
use of directed graphs to represent relations on a set is illustrated in Examples 8-10. 


EXAMPLE 8 


EXAMPLE 9 


We will study directed 
graphs extensively in 


Chapter 10. 
EXAMPLE 10 
1 2 
4 3 


FIGURE4 The 
Directed Graph 
of the Relation R. 
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The directed graph of the relation 
R={(1, 1), 0.3), @ D, 2:3), 2.4); GD, G2). 4 DI 


on the set {1, 2, 3, 4} is shown in Figure 4. < 


What are the ordered pairs in the relation R represented by the directed graph shown in Figure 5? 


Solution: The ordered pairs (x, y) in the relation are 
R= {(1,3), 1,4); @; 1), @, 2); @, 3), G, D, G, 3), 4 0; 4, 3. 


Each of these pairs corresponds to an edge of the directed graph, with (2, 2) and (3, 3) corre- 
sponding to loops. < 


The directed graph representing a relation can be used to determine whether the relation 
has various properties. For instance, a relation 1s reflexive if and only if there is a loop at every 
vertex of the directed graph, so that every ordered pair of the form (x, x) occurs in the relation. 
A relation is symmetric if and only if for every edge between distinct vertices in its digraph 
there is an edge in the opposite direction, so that (y, x) is in the relation whenever (x, y) is 
in the relation. Similarly, a relation is antisymmetric if and only if there are never two edges 
in opposite directions between distinct vertices. Finally, a relation is transitive if and only if 
whenever there is an edge from a vertex x to a vertex y and an edge from a vertex y toa 
vertex z, there is an edge from x to z (completing a triangle where each side is a directed edge 
with the correct direction). 


Remark: Note that a symmetric relation can be represented by an undirected graph, which is a 
graph where edges do not have directions. We will study undirected graphs in Chapter 10. 


Determine whether the relations for the directed graphs shown in Figure 6 are reflexive, sym- 
metric, antisymmetric, and/or transitive. 


Solution: Because there are loops at every vertex of the directed graph of R, it is reflexive. R is 
neither symmetric nor antisymmetric because there is an edge from a to b but not one from b to 
a, but there are edges in both directions connecting b and c. Finally, R is not transitive because 
there is an edge from a to b and an edge from b to c, but no edge from a to c. 


a b 
a 
b Cc c d 
(a) Directed graph of R (b) Directed graph of S$ 
FIGURE 5 The FIGURE 6 The Directed Graphs of the 
Directed Graph Relations R and S. 
of the Relation R. 
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Because loops are not present at all the vertices of the directed graph of S, this relation is not 
reflexive. It is symmetric and not antisymmetric, because every edge between distinct vertices 
is accompanied by an edge in the opposite direction. It is also not hard to see from the directed 
graph that S is not transitive, because (c, a) and (a, b) belong to S, but (c, b) does not belong 


to S. 


Exercises 


1. Represent each of these relations on {1, 2, 3} with a matrix 


(with the elements of this set listed in increasing order). 

a) {(1, 1), (1, 2), 1, 3)} 

b) {C, 2), 2, 1), @, 2), 3, 3)} 

ec) {C, 1), (1, 2), C, 3), 2, 2), 2, 3), (3, 3)} 

d) {, 3), 3, D} 

. Represent each of these relations on {1, 2,3, 4} with a 

matrix (with the elements of this set listed in increasing 

order). 

a) {(1,2), (1,3), (1,4), (2,3), (2,4), (3, 4)} 

b) (C1, 1), (1,4), 2,2), G, 3), 4, D} 

ec) {C, 2), Cd, 3), (4), @, 1), 2, 3), 2,4), 3, D, G, 2), 
(3, 4), (4, 1), (4, 2), 4, 3)} 

d) {(2, 4), B,D, G, 2), 3,4} 

. List the ordered pairs in the relations on {1, 2, 3} corre- 

sponding to these matrices (where the rows and columns 

correspond to the integers listed in increasing order). 


1 01 0 1 0 
a) |}0 1 0 b) |}O0 1 0 
101 0 1 0 
1 11 
ce) }1 0 1 
1 11 


. List the ordered pairs in the relations on {1, 2, 3, 4} corre- 
sponding to these matrices (where the rows and columns 
correspond to the integers listed in increasing order). 


1101 1110 
i oe tO 0100 
aoe a ea Dlo 011 
Ge a a 100-1 
0101 
- [1 9 1 0 
0101 
1010 


. How can the matrix representing a relation R on a set A 
be used to determine whether the relation is irreflexive? 
. How can the matrix representing a relation R on a set A 
be used to determine whether the relation is asymmetric? 
. Determine whether the relations represented by the ma- 
trices in Exercise 3 are reflexive, irreflexive, symmetric, 
antisymmetric, and/or transitive. 

. Determine whether the relations represented by the ma- 
trices in Exercise 4 are reflexive, irreflexive, symmetric, 
antisymmetric, and/or transitive. 


10. 


11. 


12. 


13. 


14. 


15. 


< 


. How many nonzero entries does the matrix representing 


the relation R on A = {1, 2,3, ..., 100} consisting of the 
first 100 positive integers have if R is 

a) {(a,b)|a> b}? b) {(a, b) |a 4b}? 

ce) {(a,b)|a=b+1}? d) {(a,b)|a=1}? 

e) {(a,b) | ab= 1}? 

How many nonzero entries does the matrix representing 
the relation R on A = {1, 2,3,..., 1000} consisting of 
the first 1000 positive integers have if R is 

a) {(a,b) | a < b}? 

b) {(a,b)|a=b+1}? 

ce) {(a,b) | a+b = 1000}? 

d) {(a,b)|a+b < 1001}? 

e) {(a,b) | a #0}? 

How can the matrix for R, the complement of the 
relation R, be found from the matrix representing R, 
when R is a relation on a finite set A? 


How can the matrix for R~!, the inverse of the 
relation R, be found from the matrix representing R, 
when R is a relation on a finite set A? 

Let R be the relation represented by the matrix 


0 1 1 
Mr=]1 1 0 
1 0 1 
Find the matrix representing 
a) RTI. b) R. c) R°. 


Let R; and R2 be relations on a set A represented by the 
matrices 


0 1 0 0 1 0 
Mr, =/1 1 1 and Mr, =/0 1 1 
1 0 0 1 1 1 
Find the matrices that represent 
a) R, UR>. b) Ry ON Ro. c) RooR,. 
d) Ri oR. e) Ri ® Ro. 
Let R be the relation represented by the matrix 


0 1 0 
Mr=|0 0 1 
1 1 0 


Find the matrices that represent 


a) R?. b) R?. c) Rt. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 
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Let R be a relation on a set A with n elements. If there 27. 28. 

are k nonzero entries in Ma, the matrix representing R, a b Q——sD 

how many nonzero entries are there in Mp-1, the matrix b 
representing R~!, the inverse of R? 

Let R be a relation on a set A with n elements. If there 

are k nonzero entries in Mr, the matrix representing R, 

how many nonzero entries are there in Mg, the matrix ° Oa Ca 3 
ppresenlatie hy elie COmplemeny OF Et 29. How can the directed graph of a relation R on a finite 


Draw the directed graphs representing each of the rela- 
tions from Exercise 1. 

Draw the directed graphs representing each of the rela- 
tions from Exercise 2. 

Draw the directed graph representing each of the relations 
from Exercise 3. 

Draw the directed graph representing each of the relations 
from Exercise 4. 

Draw the directed graph that represents the relation 


30. 


31. 


set A be used to determine whether a relation is asym- 
metric? 


How can the directed graph of a relation R on a finite 
set A be used to determine whether a relation is irreflex- 
ive? 

Determine whether the relations represented by the di- 
rected graphs shown in Exercises 23-25 are reflexive, 
irreflexive, symmetric, antisymmetric, and/or transitive. 


{(a, a), (a, b), (b,c), (c, b), (c,d), (d, a), (d, b)}. 32. Determine whether the relations represented by the di- 


In Exercises 23-28 list the ordered pairs in the relations rep- 
resented by the directed graphs. 


23. 
a 
A 
25. 


rected graphs shown in Exercises 26—28 are reflexive, ir- 
reflexive, symmetric, antisymmetric, asymmetric, and/or 
transitive. 


24. 
ts 33. Let R be a relation on a set A. Explain how to use the di- 
rected graph representing R to obtain the directed graph 
representing the inverse relation R~!. 
34. Let R be a relation on a set A. Explain how to use the di- 
b c 
26. 


rected graph representing R to obtain the directed graph 
representing the complementary relation R. 


a b a b 
<—_—_ + 35. Show that if Me is the matrix representing the relation R, 
then mM") is the matrix representing the relation R”. 
36. Given the directed graphs representing two relations, how 
can the directed graph of the union, intersection, sym- 
C d metric difference, difference, and composition of these 
c d relations be found? 


9.4 Closures of Relations 


Introduction 


A computer network has data centers in Boston, Chicago, Denver, Detroit, New York, and San 
Diego. There are direct, one-way telephone lines from Boston to Chicago, from Boston to 
Detroit, from Chicago to Detroit, from Detroit to Denver, and from New York to San Diego. 
Let R be the relation containing (a, b) if there is a telephone line from the data center in a to 
that in b. How can we determine if there is some (possibly indirect) link composed of one or 
more telephone lines from one center to another? Because not all links are direct, such as the 
link from Boston to Denver that goes through Detroit, R cannot be used directly to answer this. 
In the language of relations, R is not transitive, so it does not contain all the pairs that can be 
linked. As we will show in this section, we can find all pairs of data centers that have a link 
by constructing a transitive relation S containing R such that S is a subset of every transitive 
relation containing R. Here, S is the smallest transitive relation that contains R. This relation is 
called the transitive closure of R. 

In general, let R be a relation on a set A. R may or may not have some property P, such as 
reflexivity, symmetry, or transitivity. If there is a relation S with property P containing R such 
that S is a subset of every relation with property P containing R, then S is called the closure 
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EXAMPLE 2 


Extra 
Examples 


< 


of R with respect to P. (Note that the closure of a relation with respect to a property may not 
exist; see Exercises 15 and 35.) We will show how reflexive, symmetric, and transitive closures 
of relations can be found. 


Closures 


The relation R = {(1, 1), C1, 2), (2, 1), (, 2)} on the set A = {1, 2, 3} is not reflexive. How can 
we produce a reflexive relation containing RF that is as small as possible? This can be done by 
adding (2, 2) and (3, 3) to R, because these are the only pairs of the form (a, a) that are not in R. 
Clearly, this new relation contains R. Furthermore, any reflexive relation that contains R must 
also contain (2, 2) and (3,3). Because this relation contains R, is reflexive, and is contained 
within every reflexive relation that contains R, it is called the reflexive closure of R. 

As this example illustrates, given a relation R on a set A, the reflexive closure of R can be 
formed by adding to R all pairs of the form (a, a) with a € A, not already in R. The addition 
of these pairs produces a new relation that is reflexive, contains R, and is contained within any 
reflexive relation containing R. We see that the reflexive closure of R equals RU A, where 
A = {(a, a) | a € A} is the diagonal relation on A. (The reader should verify this.) 


What is the reflexive closure of the relation R = {(a, b) | a < b} on the set of integers? 
Solution: The reflexive closure of R is 
RUA={(a,b)| a < b} U{(a,a) | a € Z} = {(a, b) | a < D}. < 


The relation {(1, 1), (1, 2), (2, 2), (2, 3), G, 1), G, 2)} on {1, 2, 3} is not symmetric. How 
can we produce a symmetric relation that is as small as possible and contains R? To do this, 
we need only add (2, 1) and (1, 3), because these are the only pairs of the form (b, a) with 
(a, b) € R that are not in R. This new relation is symmetric and contains R. Furthermore, any 
symmetric relation that contains R must contain this new relation, because a symmetric relation 
that contains R must contain (2, 1) and (1,3). Consequently, this new relation is called the 
symmetric closure of R. 

As this example illustrates, the symmetric closure of a relation R can be constructed by 
adding all ordered pairs of the form (b, a), where (a, b) is in the relation, that are not al- 
ready present in R. Adding these pairs produces a relation that is symmetric, that contains R, 
and that is contained in any symmetric relation that contains R. The symmetric closure of a 
relation can be constructed by taking the union of a relation with its inverse (defined in the 
preamble of Exercise 26 in Section 9.1); that is, R U R-! is the symmetric closure of R, where 
R—! = {(b, a) | (a, b) € R}. The reader should verify this statement. 


What is the symmetric closure of the relation R = {(a, b) | a > b} onthe set of positive integers? 
Solution: The symmetric closure of R is the relation 
RUR! ={(a,b) |a> b}U {(b, a) |a > b} = {(a,b) |a 4D}. 


This last equality follows because R contains all ordered pairs of positive integers where the 
first element is greater than the second element and R~! contains all ordered pairs of positive 
integers where the first element is less than the second. < 


Suppose that a relation R is not transitive. How can we produce a transitive relation that 
contains R such that this new relation is contained within any transitive relation that con- 
tains R? Can the transitive closure of a relation R be produced by adding all the pairs of 
the form (a,c), where (a,b) and (b,c) are already in the relation? Consider the relation 


DEFINITION 1 


EXAMPLE 3 
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R= {d, 3), d, 4), @, 1), G, 2)} on the set {1, 2, 3, 4}. This relation is not transitive because 
it does not contain all pairs of the form (a,c) where (a, b) and (b,c) are in R. The pairs of 
this form not in R are (1, 2), (2, 3), (2, 4), and (3, 1). Adding these pairs does not produce a 
transitive relation, because the resulting relation contains (3, 1) and (1, 4) but does not contain 
(3, 4). This shows that constructing the transitive closure of a relation is more complicated than 
constructing either the reflexive or symmetric closure. The rest of this section develops algo- 
rithms for constructing transitive closures. As will be shown later in this section, the transitive 
closure of a relation can be found by adding new ordered pairs that must be present and then 
repeating this process until no new ordered pairs are needed. 


Paths in Directed Graphs 


We will see that representing relations by directed graphs helps in the construction of transitive 
closures. We now introduce some terminology that we will use for this purpose. 

A path in a directed graph is obtained by traversing along edges (in the same direction as 
indicated by the arrow on the edge). 


A path from a to b in the directed graph G is a sequence of edges (xo, x1), (%1, x2), 
(x2, X3),---, (Xn-1, Xn) in G, where n is a nonnegative integer, and x9 = a and x, = b, 
that is, a sequence of edges where the terminal vertex of an edge is the same as the initial 
vertex in the next edge in the path. This path is denoted by xo, x1, x2, ...,Xn—1, Xn and has 
length n. We view the empty set of edges as a path of length zero from a to a. A path of 
length n > | that begins and ends at the same vertex is called a circuit or cycle. 


A path in a directed graph can pass through a vertex more than once. Moreover, an edge in 
a directed graph can occur more than once in a path. 


Which of the following are paths in the directed graph shown in Figure |: a, b, e, d; a, e, c,d, b; 
b,a,c,b,a,a,b; d,c;c,b,a; e,b, a,b, a,b, e? What are the lengths of those that are paths? 
Which of the paths in this list are circuits? 


Solution: Because each of (a, b), (b, e), and (e, d) is an edge, a, b, e, d is a path of length three. 
Because (c,d) is not an edge, a, e, c,d, b is not a path. Also, b, a,c, b,a,a,b is a path of 
length six because (b, a), (a, c), (c, b), (b, a), (a, a), and (a, b) are all edges. We see that d, c 
is a path of length one, because (d, c) is an edge. Also c, b, a is a path of length two, because 
(c, b) and (b, a) are edges. All of (e, b), (b, a), (a, b), (b, a), (a, b), and (b, e) are edges, so 
e, b, a, b, a, b, eis a path of length six. 

The two paths b,a,c,b,a,a,b and e, b,a, b, a, b, e are circuits because they begin and 
end at the same vertex. The paths a, b, e, d; c, b, a; and d, c are not circuits. 


FIGURE 1 A Directed Graph. 
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The term path also applies to relations. Carrying over the definition from directed graphs to 
relations, there is a path from a to b in R if there is a sequence of elements a, x1, x2,...,Xn—1,0 
with (a, x,) € R, (x1, x2) € R,..., and (x,_1, b) € R. Theorem | can be obtained from the 
definition of a path in a relation. 


Let R be a relation on a set A. There is a path of length n, where n is a positive integer, from 
a to b if and only if (a, b) € R”. 


Proof: We will use mathematical induction. By definition, there is a path from a to b of length 
one if and only if (a, b) € R, so the theorem is true when n = 1. 

Assume that the theorem is true for the positive integer n. This is the inductive hypothesis. 
There is a path of length n + | from a to b if and only if there is an element c € A such that 
there is a path of length one from a to c, so (a,c) € R, and a path of length n from c to b, 
that is, (c, b) € R”. Consequently, by the inductive hypothesis, there is a path of length n + 1 
from a to b if and only if there is an element c with (a,c) € R and (c, b) € R”. But there 
is such an element if and only if (a,b) € R"*!. Therefore, there is a path of length n + 1 
from a to b if and only if (a, b) € R"*!. This completes the proof. 


Transitive Closures 


We now show that finding the transitive closure of a relation is equivalent to determining which 
pairs of vertices in the associated directed graph are connected by a path. With this in mind, we 
define a new relation. 


Let R be a relation on a set A. The connectivity relation R* consists of the pairs (a, b) such 
that there is a path of length at least one from a to b in R. 


Because R” consists of the pairs (a, b) such that there is a path of length n from a to b, it follows 
that R* is the union of all the sets R”. In other words, 


[oe] 


R= U R”. 


n=l 
The connectivity relation is useful in many models. 


Let R be the relation on the set of all people in the world that contains (a, b) if a has met b. 
What is R”, where v is a positive integer greater than one? What is R*? 


Solution: The relation R? contains (a, b) if there is apersonc such that (a,c) € Rand(c, b) € R, 
that is, if there is a person c such that a has met c and c has met b. Similarly, R” consists of 
those pairs (a, b) such that there are people x;, x2,...,X,— 1 such that a has met x1, x; has 
met x2,..., and x,_; has met D. 

The relation R* contains (a, b) if there is a sequence of people, starting with a and ending 
with b, such that each person in the sequence has met the next person in the sequence. (There 
are many interesting conjectures about R*. Do you think that this connectivity relation includes 
the pair with you as the first element and the president of Mongolia as the second element? We 
will use graphs to model this application in Chapter 10.) < 
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Let R be the relation on the set of all subway stops in New York City that contains (a, b) if it is 
possible to travel from stop a to stop b without changing trains. What is R” when n is a positive 
integer? What is R*? 


Solution: The relation R” contains (a, b) if itis possible to travel from stop a to stop b by making 
at most n — 1 changes of trains. The relation R* consists of the ordered pairs (a, b) where it is 
possible to travel from stop a to stop b making as many changes of trains as necessary. (The 
reader should verify these statements.) 


Let R be the relation on the set of all states in the United States that contains (a, b) if state a 
and state b have a common border. What is R", where n is a positive integer? What is R*? 


Solution: The relation R” consists of the pairs (a, b), where it is possible to go from state a to 
state b by crossing exactly n state borders. R* consists of the ordered pairs (a, b), where it is 
possible to go from state a to state b crossing as many borders as necessary. (The reader should 
verify these statements.) The only ordered pairs not in R* are those containing states that are not 
connected to the continental United States (i.e., those pairs containing Alaska or Hawaii). <4 


Theorem 2 shows that the transitive closure of a relation and the associated connectivity 
relation are the same. 


The transitive closure of a relation R equals the connectivity relation R*. 


Proof: Note that R* contains R by definition. To show that R* is the transitive closure of R 
we must also show that R* is transitive and that R* C S whenever S is a transitive relation that 
contains R. 

First, we show that R* is transitive. If (a,b) € R* and (b,c) € R*, then there are paths 
from a to b and from b to c in R. We obtain a path from a to c by starting with the path 
from a to b and following it with the path from b to c. Hence, (a, c) € R*. It follows that R* is 
transitive. 

Now suppose that S is a transitive relation containing R. Because S is transitive, S” also is 
transitive (the reader should verify this) and S” C S (by Theorem | of Section 9.1). Furthermore, 
because 


[o.@) 
=| 1S 
k=1 


and S* CS, it follows that S* C S. Now note that if R C S, then R* C S*, because any 
path in R is also a path in S. Consequently, R* C S* C S. Thus, any transitive relation that 
contains R must also contain R*. Therefore, R* is the transitive closure of R. <j 


Now that we know that the transitive closure equals the connectivity relation, we turn our 
attention to the problem of computing this relation. We do not need to examine arbitrarily long 
paths to determine whether there is a path between two vertices in a finite directed graph. As 
Lemma | shows, it is sufficient to examine paths containing no more than n edges, where n is 
the number of elements in the set. 


Let A be a set with n elements, and let R be a relation on A. If there is a path of length at 
least one in R from a to b, then there is such a path with length not exceeding n. Moreover, 
when a # J, if there is a path of length at least one in R from a to b, then there is such a path 
with length not exceeding n — 1. 
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FIGURE 2 Producing a Path with Length Not Exceeding n. 


Proof: Suppose there is a path from a to b in R. Let m be the length of the shortest such path. 
Suppose that xo, x1, x2,...,%Xm—1,Xm, where x9 = a and x» = b, is such a path. 

Suppose thata = bandthatm > n,sothatm > n+ 1. By the pigeonhole principle, because 
there are n vertices in A, among the m vertices x9, x1,...,Xm—1, at least two are equal (see 
Figure 2). 

Suppose that x; = x; with 0 <i < j <mW—1. Then the path contains a circuit from 
x; to itself. This circuit can be deleted from the path from a to b, leaving a path, namely, 


Xs X 1, 0+ +5 Xi, Xj41,+++sXm—1, Xm, from a to b of shorter length. Hence, the path of shortest 
length must have length less than or equal to n. 
The case where a # b is left as an exercise for the reader. | 


From Lemma 1, we see that the transitive closure of R is the union of R, R?, R3, ae 
and R”. This follows because there is a path in R* between two vertices if and only if there is a 
path between these vertices in R', for some positive integer i with i < n. Because 


R* = RUR*UR?U.---U R® 


and the zero—one matrix representing a union of relations is the join of the zero—one matrices of 
these relations, the zero—one matrix for the transitive closure is the join of the zero—one matrices 
of the first n powers of the zero—one matrix of R. 


Let Mp be the zero—one matrix of the relation R on a set with n elements. Then the zero—one 
matrix of the transitive closure R* is 


Mr =Mra VME! vMEly...v Me, 


EXAMPLE 7 | Find the zero—one matrix of the transitive closure of the relation R where 


Lt & 4 
Mr=|0 1 0 
1 1.0 


Solution: By Theorem 3, it follows that the zero—one matrix of R* is 
Mr = Mr VME) Vv Mel. 


Because 


Links 


Links 
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it follows that 


1 0 1 1 1 1 1 1 1 1 1 1 
Mre=]0 1 OJ vjJO 1 O}v}jO 1 OJ =]0 1 O 
1 1 0 1 11 1 1 1 1 1 1 


< 


Theorem 3 can be used as a basis for an algorithm for computing the matrix of the 
relation R*. To find this matrix, the successive Boolean powers of Mr, up to the nth power, are 
computed. As each power is calculated, its join with the join of all smaller powers is formed. 
When this is done with the nth power, the matrix for R* has been found. This procedure is 
displayed as Algorithm 1. 


ALGORITHM 1 A Procedure for Computing the Transitive Closure. 


procedure transitive closure (Mr : zero—one n x n matrix) 
A:= Mer 
B:=A 
fori :=2 ton 
A:=A©Mapr 
B:=BvaA 
return B{B is the zero—one matrix for R*} 


We can easily find the number of bit operations used by Algorithm 1 to determine the 
transitive closure of a relation. Computing the Boolean powers Mr, Mul, ..., Mn requires 
thatn — | Boolean products of n x n zero—one matrices be found. Each of these Boolean products 
can be found using n?(2n — 1) bit operations. Hence, these products can be computed using 
n2(2n — 1)(n— 1) bit operations. 

To find Mr« from the n Boolean powers of Mr, n — 1 joins of zero—one matrices need 
to be found. Computing each of these joins uses n* bit operations. Hence, (n — 1)n bit op- 
erations are used in this part of the computation. Therefore, when Algorithm | is used, the 
matrix of the transitive closure of a relation on a set with n elements can be found using 
n?(2n — 1)(n— 1) + (n— 1)n? = 2n3(n — 1), which is O(n*) bit operations. The remainder of 
this section describes a more efficient algorithm for finding transitive closures. 


Warshall’s Algorithm 


Warshall’s algorithm, named after Stephen Warshall, who described this algorithm in 1960, is 
an efficient method for computing the transitive closure of a relation. Algorithm | can find the 
transitive closure of a relation on a set with n elements using 2n3(n — 1) bit operations. However, 
the transitive closure can be found by Warshall’s algorithm using only 2n? bit operations. 


Remark: Warshall’s algorithm is sometimes called the Roy—Warshall algorithm, because 
Bernard Roy described this algorithm in 1959. 


Suppose that R is a relation on a set with n elements. Let v1}, v2,..., Vv, be an arbitrary 
listing of these n elements. The concept of the interior vertices of a path is used in Warshall’s 
algorithm. If a, x1, x2,...,Xm_—1, b is a path, its interior vertices are x1, X2,...,X—1, that 
is, all the vertices of the path that occur somewhere other than as the first and last vertices in 
the path. For instance, the interior vertices of a path a,c,d, f,g,h,b, j in a directed graph 
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<< 


d ic 


FIGURE 3 
The Directed 
Graph of the 
Relation R. 


are c,d, f, g,h, and b. The interior vertices of a,c, d,a, f, b are c,d, a, and f. (Note that the 
first vertex in the path is not an interior vertex unless it is visited again by the path, except as 
the last vertex. Similarly, the last vertex in the path is not an interior vertex unless it was visited 
previously by the path, except as the first vertex.) 

Warshall’s algorithm is based on the construction of a sequence of zero—one matrices. These 
matrices are Wo, W,,..., W,,, where Wo = Mp is the zero—one matrix of this relation, and 
W,= [wi], where wi = 1 if there is a path from v; to v; such that all the interior vertices of 
this path are in the set {v1, v2,..., Ve} (the first k vertices in the list) and is 0 otherwise. (The 
first and last vertices in the path may be outside the set of the first k vertices in the list.) Note 
that W,, = Mp«, because the (i, 7)th entry of Mp: is | if and only if there is a path from v; 
to v;, with all interior vertices in the set {vj, v2, ..., Vn} (but these are the only vertices in the 
directed graph). Example 8 illustrates what the matrix W, represents. 


Let R be the relation with directed graph shown in Figure 3. Let a, b, c, d be a listing of the 
elements of the set. Find the matrices Wo, W,, W2, W3, and Wy. The matrix W, is the transitive 
closure of R. 


Solution: Let vj = a, v2 = b, v3 = c, and v4 = d. Wo is the matrix of the relation. Hence, 


Wo = 


orrco 
oooo 
a) 
oror 


W, has 1 as its (é, j)th entry if there is a path from v; to v; that has only vj = a as an interior 
vertex. Note that all paths of length one can still be used because they have no interior vertices. 
Also, there is now an allowable path from b to d, namely, b, a, d. Hence, 


Oo @ 1 
Nt Of A 
Wael a og: 4 ke 
lo o 1 o| 


W> has 1 as its (7, j)th entry if there is a path from v; to v; that has only vj = a and/or v2 = b 
as its interior vertices, if any. Because there are no edges that have b as a terminal vertex, no 
new paths are obtained when we permit b to be an interior vertex. Hence, W2 = W. 


STEPHEN WARSHALL (1935-2006) Stephen Warshall, born in New York City, went to public school in 
Brooklyn. He attended Harvard University, receiving his degree in mathematics in 1956. He never received an 
advanced degree, because at that time no programs were available in his areas of interest. However, he took 
graduate courses at several different universities and contributed to the development of computer science and 
software engineering. 

After graduating from Harvard, Warshall worked at ORO (Operation Research Office), which was set 
up by Johns Hopkins to do research and development for the U.S. Army. In 1958 he left ORO to take a 
position at a company called Technical Operations, where he helped build a research and development labo- 
ratory for military software projects. In 1961 he left Technical Operations to found Massachusetts Computer 


Associates. Later, this company became part of Applied Data Research (ADR). After the merger, Warshall sat on the board of 
directors of ADR and managed a variety of projects and organizations. He retired from ADR in 1982. 

During his career Warshall carried out research and development in operating systems, compiler design, language design, and 
operations research. In the 1971-1972 academic year he presented lectures on software engineering at French universities. There is 
an interesting anecdote about his proof that the transitive closure algorithm, now known as Warshall’s algorithm, is correct. He and 
a colleague at Technical Operations bet a bottle of rum on who first could determine whether this algorithm always works. Warshall 
came up with his proof overnight, winning the bet and the rum, which he shared with the loser of the bet. Because Warshall did not 
like sitting at a desk, he did much of his creative work in unconventional places, such as on a sailboat in the Indian Ocean or in a 
Greek lemon orchard. 
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W; has | as its (i, j)th entry if there is a path from v; to v; that has only vj = a, v2 = b, 
and/or v3 = c as its interior vertices, if any. We now have paths from d to a, namely, d, c, a, 
and from d to d, namely, d, c, d. Hence, 


W3= 


—— = © 
oocoeo 
i) 
ee ee 


Finally, W4 has 1 as its (i, j)th entry if there is a path from v; to v; that has vj = a, v2 = b, 
v3 = c, and/or v4 = d as interior vertices, if any. Because these are all the vertices of the graph, 
this entry is 1 if and only if there is a path from v; to v;. Hence, 


Ws = 


RP Ree 
ooco 
RP RRR 
PR Re 


This last matrix, W4, is the matrix of the transitive closure. < 


Warshall’s algorithm computes Mr: by efficiently computing Wo = Mr, Wi, W2,..., 
W,, = Mpe:. This observation shows that we can compute W, directly from W,;_1: There is a 
path from v; to v; with no vertices other than v,, v2, ..., vg as interior vertices if and only if 
either there is a path from v; to vj with its interior vertices among the first k — 1 vertices in the 
list, or there are paths from v; to vz and from v, to v; that have interior vertices only among the 
first k — 1 vertices in the list. That is, either a path from v; to v; already existed before vy; was 
permitted as an interior vertex, or allowing vz as an interior vertex produces a path that goes 
from v; to vz; and then from v,z to v;. These two cases are shown in Figure 4. 


The first type of path exists if and only if wee = |, and the second type of path exists if 
k-1] [k-1] ylkl 


and only if both wh k and Wp; are 1. Hence, w; j 


[k—1] [k—1] 
Wie and We 


is | if and only if either wikis 1 or both 


ij 


are 1. This gives us Lemma 2. 


ee a ge 


All interior vertices 
in {V,V,..-, Ye} 


i All interior vertices J 
in {Vp,V2,.--, V-1) 


FIGURE 4 Adding v; to the Set of 
Allowable Interior Vertices. 


606 9/ Relations 


LEMMA 2 Let W; = [with be the zero—one matrix that has a | in its (i, 7)th position if and only if there 
is a path from v; to v; with interior vertices from the set {v), v2,..., ve}. Then 


[kK] | [k—1] 


[k—-1] [k—1] 
ae 


Ww V Wiz A Wij Vy, 


whenever i, j, and k are positive integers not exceeding n. 


Lemma 2 gives us the means to compute efficiently the matrices W;, k = 1, 2,...,n. We 
display the pseudocode for Warshall’s algorithm, using Lemma 2, as Algorithm 2. 


ALGORITHM 2 Warshall Algorithm. 


procedure Warshall (Mr :n x n zero—one matrix) 


W:=Me 
fork :=1ton 
fori:=1ton 
for j:=1ton 


Wij : = Wij V (Wik A Wkj) 
return W{W = [w;;] is Mr«} 


The computational compleny of Warshall’s algorithm can easily be computed in terms of 
bit operations. To find the entry wit from the entries we, wie! , and Wei using Lemma 
2 requires two bit operations. To find all n entries of Wi from those of Wie requires 2n? 
bit operations. Because Warshall’s algorithm begins with Wo = Mr and computes the se- 
quence Of n peO-one matrices W,;, Wo, ..., W, = Mr, the total number of bit operations used 
isn-2n* = 2n°. 


Exercises 
1. Let R be the relation on the set {0, 1, 2, 3} containing 7. 
the ordered pairs (0, 1), (1, 1), C1, 2), (2, 0), (2, 2), and eG b 


(3, 0). Find the 

a) reflexive closure of R. b) symmetric closure of R. 
2. Let R be the relation {(a, b) | a £ b} on the set of inte- 

gers. What is the reflexive closure of R? 


3. Let R be the relation {(a, b) | a divides b} on the set of 


integers. What is the symmetric closure of R? c d 
4. How can the directed graph representing the reflexive clo- 8. How can the directed graph representing the symmetric 
sure of a relation on a finite set be constructed from the closure of a relation on a finite set be constructed from 
directed graph of the relation? the directed graph for this relation? 
In Exercises 5—7 draw the directed graph of the reflexive clo- 9. Find the directed graphs of the symmetric closures of the 
sure of the relations with the directed graph shown. relations with directed graphs shown in Exercises 5-7. 
5. 6. 10. Find the smallest relation containing the relation in Ex- 


b 


a b ample 2 that is both reflexive and symmetric. 


11. Find the directed graph of the smallest relation that is 
both reflexive and symmetric that contains each of the 
relations with directed graphs shown in Exercises 5—7. 


12. Suppose that the relation R on the finite set A is rep- 
resented by the matrix Mr. Show that the matrix that 
represents the reflexive closure of R is Mr V I,. 


io) 
a) 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


Suppose that the relation R on the finite set A is rep- 
resented by the matrix Mr. Show that the matrix that 
represents the symmetric closure of R is Mr V Mh. 
Show that the closure of a relation R with respect to a 
property P, if it exists, is the intersection of all the rela- 
tions with property P that contain R. 

When is it possible to define the “irreflexive closure” 
of a relation R, that is, a relation that contains R, is ir- 
reflexive, and is contained in every irreflexive relation 
that contains R? 

Determine whether these sequences of vertices are paths 
in this directed graph. 


Find all circuits of length three in the directed graph in 
Exercise 16. 

Determine whether there is a path in the directed graph in 
Exercise 16 beginning at the first vertex given and ending 
at the second vertex given. 


a) a,b b) b,a c) b,b 
d) a,e e) b,d f) c,d 
g) d,d h) e,a i) e,c 


Let R be the relation on the set {1, 2, 3, 4, 5} containing 
the ordered pairs (1, 3), (2, 4), (3, 1), (3, 5), (4, 3), (5, 1), 
(5, 2), and (5, 4). Find 

a) R. b) R?. c) Rt. 

d) R. e) R°. f) R*. 

Let R be the relation that contains the pair (a, b) if a 
and b are cities such that there is a direct non-stop airline 
flight from a to b. When is (a, b) in 


a) R2? b) R?? c) R*? 


Let R be the relation on the set of all students contain- 
ing the ordered pair (a, b) if a and b are in at least one 
common class and a 4 b. When is (a, b) in 

a) R?? b) R3? c) R*? 


Suppose that the relation R is reflexive. Show that R* is 
reflexive. 

Suppose that the relation R is symmetric. Show that R* 
is symmetric. 

Suppose that the relation R is irreflexive. Is the 
relation R? necessarily irreflexive? 


Equivalence Relations 


25. 


26. 


27. 


28. 


29. 


30. 
31. 


* 32. 


33. 


34. 


35. 
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Use Algorithm | to find the transitive closures of these 

relations on {1, 2, 3, 4}. 

a) {(1, 2), (2,1), 2,3), 3,4), 4. D} 

b) {2, 1), 2,3), GB, 1), 3,4), 4D, 4, 3} 

ce) {(, 2), (1,3), 1,4), 2,3), 2,4), 3, D} 

d) {(1, 1), 1.4), @, 1), 2,3), 3, D, G, 2), 3,4), 4, 2)} 

Use Algorithm | to find the transitive closures of these 

relations on {a, b,c, d, e}. 

a) {(a,c), (b, d), (c, a), (d, b), (e, d)} 

b) {(, c), (b, e), (c, e), (d, a), (e, b), (e, ©)} 

c) {(a, b), a,c), G,e), (©, a), (b,€), (c, a), (c,b), , a), 
(e, d)} 

d) {(a, e), (b, a), (b, d), (c,d), (d, a), (d,c), (@, 4), (e, 5), 
(e, c), (e, e)} 

Use Warshall’s algorithm to find the transitive closures 

of the relations in Exercise 25. 

Use Warshall’s algorithm to find the transitive closures 

of the relations in Exercise 26. 

Find the smallest relation containing the relation 

{(, 2), (1, 4), @, 3), (4, 1)} that is 

a) reflexive and transitive. 

b) symmetric and transitive. 

c) reflexive, symmetric, and transitive. 

Finish the proof of the case when a 4 b in Lemma 1. 

Algorithms have been devised that use O(n?) bit opera- 

tions to compute the Boolean product of twon x n zero— 

one matrices. Assuming that these algorithms can be used, 

give big-O estimates for the number of bit operations us- 

ing Algorithm | and using Warshall’s algorithm to find the 

transitive closure of a relation on a set with n elements. 

Devise an algorithm using the concept of interior vertices 

in a path to find the length of the shortest path between 

two vertices in a directed graph, if such a path exists. 

Adapt Algorithm 1 to find the reflexive closure of the 

transitive closure of a relation on a set with n elements. 


Adapt Warshall’s algorithm to find the reflexive closure of 
the transitive closure of arelation ona set withn elements. 


Show that the closure with respect to the property P of 
the relation R = {(0, 0), (0, 1), C1, 1), (2, 2)} on the set 
{0, 1, 2} does not exist if P is the property 

a) “is not reflexive.” 

b) “has an odd number of elements.” 


Introduction 


In some programming languages the names of variables can contain an unlimited number of 
characters. However, there is a limit on the number of characters that are checked when a 
compiler determines whether two variables are equal. For instance, in traditional C, only the 
first eight characters of a variable name are checked by the compiler. (These characters are 
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Links 
DEFINITION 1 


Equivalence relations are 
important in every branch 
of mathematics! 


DEFINITION 2 


EXAMPLE 1 


EXAMPLE 2 


uppercase or lowercase letters, digits, or underscores.) Consequently, the compiler considers 
strings longer than eight characters that agree in their first eight characters the same. Let R be 
the relation on the set of strings of characters such that s Rt, where s and ¢ are two strings, if s 
and ¢ are at least eight characters long and the first eight characters of s and ft agree, or s = f¢. It 
is easy to see that R is reflexive, symmetric, and transitive. Moreover, R divides the set of all 
strings into classes, where all strings in a particular class are considered the same by a compiler 
for traditional C. 

The integers a and b are related by the “congruence modulo 4” relation when 4 
divides a — b. We will show later that this relation is reflexive, symmetric, and transitive. It 
is not hard to see that a is related to b if and only if a and b have the same remainder when 
divided by 4. It follows that this relation splits the set of integers into four different classes. 
When we care only what remainder an integer leaves when it is divided by 4, we need only 
know which class it is in, not its particular value. 

These two relations, R and congruence modulo 4, are examples of equivalence relations, 
namely, relations that are reflexive, symmetric, and transitive. In this section we will show that 
such relations split sets into disjoint classes of equivalent elements. Equivalence relations arise 
whenever we care only whether an element of a set is in a certain class of elements, instead of 
caring about its particular identity. 


Equivalence Relations 


In this section we will study relations with a particular combination of properties that allows 
them to be used to relate objects that are similar in some way. 


A relation on a set A is called an equivalence relation if it is reflexive, symmetric, and 
transitive. 


Equivalence relations are important throughout mathematics and computer science. One 
reason for this is that in an equivalence relation, when two elements are related it makes sense 
to say they are equivalent. 


Two elements a and b that are related by an equivalence relation are called equivalent. The 
notation a ~ b is often used to denote that a and b are equivalent elements with respect to a 
particular equivalence relation. 


For the notion of equivalent elements to make sense, every element should be equivalent to 
itself, as the reflexive property guarantees for an equivalence relation. It makes sense to say 
that a and b are related (not just that a is related to b) by an equivalence relation, because 
when a is related to b, by the symmetric property, D is related to a. Furthermore, because an 
equivalence relation is transitive, if a and b are equivalent and b and c are equivalent, it follows 
that a and c are equivalent. 

Examples |-5 illustrate the notion of an equivalence relation. 


Let R be the relation on the set of integers such that aRb if and only if a = b ora = —b. In 
Section 9.1 we showed that R is reflexive, symmetric, and transitive. It follows that R is an 
equivalence relation. < 


Let R be the relation on the set of real numbers such that a Rb if and only if a — D is an integer. 
Is R an equivalence relation? 


Extra pe 
Examples “2 


EXAMPLE 3 


EXAMPLE 4 


EXAMPLE 5 
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Solution: Because a — a = 0 1s an integer for all real numbers a, aRa for all real numbers a. 
Hence, R is reflexive. Now suppose that aRb. Then a — b is an integer, so b —a is also an 
integer. Hence, bRa. It follows that R is symmetric. If aRb and bRc, then a—b and b—c 
are integers. Therefore, a—c = (a—b) + (b—c) is also an integer. Hence, aRc. Thus, R is 
transitive. Consequently, R is an equivalence relation. 


One of the most widely used equivalence relations is congruence modulo m, where m is an 
integer greater than 1. 


Congruence Modulo m_ Let m be an integer with m > 1. Show that the relation 
R = {(a, b) |a = b (mod m)} 


is an equivalence relation on the set of integers. 


Solution: Recall from Section 4.1 that a = b (mod m) if and only if m divides a — b. Note 
that a — a = 0 is divisible by m, because 0 = 0- m. Hence, a = a (mod m), so congruence 
modulo m is reflexive. Now suppose that a = b (mod m). Then a — b is divisible by m, so 
a—b=km, where k is an integer. It follows that b — a = (—k)m, so b = a (mod m). Hence, 
congruence modulo m is symmetric. Next, suppose that a = b (mod m) and b = c (mod m). 
Then m divides both a — b and b — c. Therefore, there are integers k and/ witha — b = km and 
b —c =Im. Adding these two equations shows that a —c = (a —b) + (b-—c) =km4+Im= 
(k + 1)m. Thus, a = c (mod m). Therefore, congruence modulo m is transitive. It follows that 
congruence modulo m is an equivalence relation. 


Suppose that R is the relation on the set of strings of English letters such that aRb if and only 
if 1(a) = 1(b), where /(x) is the length of the string x. Is R an equivalence relation? 


Solution: Because I(a) = l(a), it follows that aRa whenever a is a string, so that R is reflex- 
ive. Next, suppose that aRb, so that /(a) = 1(b). Then bRa, because /(b) = l(a). Hence, R 
is symmetric. Finally, suppose that aRb and bRc. Then /(a) = /1(b) and 1(b) = l(c). Hence, 
l(a) =I(c), so aRc. Consequently, R is transitive. Because R is reflexive, symmetric, and 
transitive, it is an equivalence relation. 


Let n be a positive integer and S a set of strings. Suppose that R, is the relation on S such 
that s R,,t if and only if s = f, or both s and ¢ have at least n characters and the first n characters 
of s and f are the same. That is, a string of fewer than n characters is related only to itself; a 
string s with at least n characters is related to a string f¢ if and only if ¢ has at least n characters 
and t begins with the n characters at the start of s. For example, let n = 3 and let S be the set 
of all bit strings. Then s R3¢ either when s = ¢ or both s and ¢ are bit strings of length 3 or more 
that begin with the same three bits. For instance, 01. R301 and 00111300101, but 01 R3010 
and 01011 R301110. 

Show that for every set S of strings and every positive integer n, R, is an equivalence 
relation on S. 


Solution: The relation R, is reflexive because s = s, so that sR, s whenever s is a string in S. 
If sR, t, then either s =f or s and ¢ are both at least n characters long that begin with the 
same n characters. This means that tR, s. We conclude that R, is symmetric. 

Now suppose that sR,¢ and tR,u. Then either s = ¢ or s and ¢ are at least n characters 
long and s and t begin with the same n characters, and either tf = u or t and u are at least n 
characters long and t and u begin with the same n characters. From this, we can deduce that 
either s = u or both s and u are n characters long and s and u begin with the same n characters 
(because in this case we know that s, t, and uw are all at least n characters long and both s and u 
begin with the same n characters as f does). Consequently, R,, is transitive. It follows that Ry is 
an equivalence relation. 
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In Examples 6 and 7 we look at two relations that are not equivalence relations. 


Show that the “divides” relation is the set of positive integers in not an equivalence relation. 


Solution: By Examples 9 and 15 in Section 9.1, we know that the “divides” relation is reflex- 
ive and transitive. However, by Example 12 in Section 9.1, we know that this relation is not 
symmetric (for instance, 2 | 4 but 4 / 2). We conclude that the “divides” relation on the set of 
positive integers is not an equivalence relation. 


Let R be the relation on the set of real numbers such that x R y if and only if x and y are real 
numbers that differ by less than 1, that is |x — y| < 1. Show that R is not an equivalence relation. 


Solution: R is reflexive because |x — x| = 0 < 1 whenever x € R. R is symmetric, for if xR y, 
where x and y are real numbers, then |x — y| < 1, which tells us that | y — x| = |x — y| < 1, so 
that yRx. However, R is not an equivalence relation because it is not transitive. Take x = 2.8, 
y=1.9, and z=1.1, so that |x —y| = |2.8-—1.9,/=09 <1, Jy—zl=|19-1.1)= 
0.8 < 1, but |x — z| = |2.8 — 1.1] = 1.7 > 1. That is, 2.8R1.9,1.9R1.1,but2.8R1.1. < 


Equivalence Classes 


Let A be the set of all students in your school who graduated from high school. Consider the 
relation R on A that consists of all pairs (x, y), where x and y graduated from the same high 
school. Given a student x, we can form the set of all students equivalent to x with respect to R. 
This set consists of all students who graduated from the same high school as x did. This subset 
of A is called an equivalence class of the relation. 


Let R be an equivalence relation on a set A. The set of all elements that are related to an 
element a of A is called the equivalence class of a. The equivalence class of a with respect 
to R is denoted by [a]r. When only one relation is under consideration, we can delete the 
subscript R and write [a] for this equivalence class. 


In other words, if R is an equivalence relation on a set A, the equivalence class of the 
element a is 


[a]r = {s | (a, 5) € R}. 


If b € [a], then D is called a representative of this equivalence class. Any element of a class 
can be used as a representative of this class. That is, there is nothing special about the particular 
element chosen as the representative of the class. 


What is the equivalence class of an integer for the equivalence relation of Example 1? 
Solution: Because an integer is equivalent to itself and its negative in this equivalence relation, 


it follows that [a] = {—a, a}. This set contains two distinct integers unless a = 0. For instance, 


[7] = {-—7, 7}, [—5] = {-—5, 5}, and [0] = {0}. < 


What are the equivalence classes of 0 and | for congruence modulo 4? 


Solution: The equivalence class of 0 contains all integers a such that a = 0 (mod 4). The integers 
in this class are those divisible by 4. Hence, the equivalence class of 0 for this relation is 


(O} = {e..) 8, =4, 0,4; 8, ..}. 


EXAMPLE 10 


EXAMPLE 11 
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The equivalence class of | contains all the integers a such that a = | (mod 4). The integers 
in this class are those that have a remainder of 1 when divided by 4. Hence, the equivalence 
class of | for this relation is 


[1] ={...,—-7, -3, 1,5,9,..-}. S| 


In Example 9 the equivalence classes of 0 and 1 with respect to congruence modulo 4 
were found. Example 9 can easily be generalized, replacing 4 with any positive integer m. 
The equivalence classes of the relation congruence modulo m are called the congruence 
classes modulo m. The congruence class of an integer a modulo m is denoted by [a]m, so 
[a]m ={...,a@-—2m,a—m,a,a+m,a+2m,...}. For instance, from Example 9 it follows 
that [O]4 = {..., —8, —4,0, 4, 8,...} and [1]4 = {..., -7, -3, 1,5, 9,...}. 


What is the equivalence class of the string 0111 with respect to the equivalence relation R3 from 
Example 5 on the set of all bit strings? (Recall that s R3¢ if and only if s and ¢ are bit strings 
with s = f or s and ¢ are strings of at least three bits that start with the same three bits.) 


Solution: The bit strings equivalent to 0111 are the bit strings with at least three bits that begin 
with 011. These are the bit strings 011, 0110, 0111, 01100, 01101, 01110, 01111, and so on. 
Consequently, 


[O11]}r,; = {011, 0110, 0111, 01100, 01101, 01110, O1111,...}. < 


Identifiers in the C Programming Language In the C programming language, an identifier 
is the name of a variable, a function, or another type of entity. Each identifier is a nonempty 
string of characters where each character is a lowercase or an uppercase English letter, a digit, 
or an underscore, and the first character is a lowercase or an uppercase English letter. Identifiers 
can be any length. This allows developers to use as many characters as they want to name an 
entity, such as a variable. However, for compilers for some versions of C, there is a limit on the 
number of characters checked when two names are compared to see whether they refer to the 
same thing. For example, Standard C compilers consider two identifiers the same when they 
agree in their first 31 characters. Consequently, developers must be careful not to use identifiers 
with the same initial 31 characters for different things. We see that two identifiers are considered 
the same when they are related by the relation R3, in Example 5. Using Example 5, we know 
that R31, on the set of all identifiers in Standard C, is an equivalence relation. 

What are the equivalence classes of each of the identifiers Number_of_tropical_ 
storms, Number_of_named_tropical_storms, and Number_of_named_tropical_storms_in_the 
Atlantic_in_2005? 


Solution: Note that when an identifier is less than 31 characters long, by the definition of R31, 
its equivalence class contains only itself. Because the identifier Number_of_tropical_storms is 
25 characters long, its equivalence class contains exactly one element, namely, itself. 

The identifier Number_of_named_tropical_storms is exactly 31 characters long. An identi- 
fier is equivalent to it when it starts with these same 31 characters. Consequently, every identifier 
at least 31 characters long that starts with Number_of_named_tropical_storms is equivalent to 
this identifier. It follows that the equivalence class of Number_of_named_tropical_storms is the 
set of all identifiers that begin with the 31 characters Number_of_named_tropical_storms. 

An identifier is equivalent to the Number_of_named_tropical_storms_in_the_Atlantic_in 
2005 if and only if it begins with its first 31 characters. Because these characters 
are Number_of_named_tropical_storms, we see that an identifier is equivalent to Num- 
ber_of_named_tropical_storms_in_the_Atlantic_in_2005 if and only if it is equivalent to Num- 
ber_of_named_tropical_storms. It follows that these last two identifiers have the same equiva- 
lence class. 
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THEOREM 1 


Recall that an index set is 
a set whose members 
label, or index, the 
elements of a set. 


Equivalence Classes and Partitions 


Let A be the set of students at your school who are majoring in exactly one subject, and let R 
be the relation on A consisting of pairs (x, y), where x and y are students with the same major. 
Then R is an equivalence relation, as the reader should verify. We can see that R splits all 
students in A into a collection of disjoint subsets, where each subset contains students with 
a specified major. For instance, one subset contains all students majoring (just) in computer 
science, and a second subset contains all students majoring in history. Furthermore, these sub- 
sets are equivalence classes of R. This example illustrates how the equivalence classes of an 
equivalence relation partition a set into disjoint, nonempty subsets. We will make these notions 
more precise in the following discussion. 

Let R be a relation on the set A. Theorem | shows that the equivalence classes of two 
elements of A are either identical or disjoint. 


Let R be an equivalence relation on a set A. These statements for elements a and b of A are 
equivalent: 


@ aRb_ (ii) [aJ=[5] ~— aid) [a] ND] FB 


Proof: We first show that (i) implies (ii). Assume that aRb. We will prove that [a] = [b] by 
showing [a] C [b] and [b] € [a]. Suppose c € [a]. ThenaRc. Because a Rb and R is symmetric, 
we know that bRa. Furthermore, because R is transitive and bRa and akc, it follows that bRce. 
Hence, c € [b]. This shows that [a] C [b]. The proof that [b] C [a] is similar; it is left as an 
exercise for the reader. 

Second, we will show that (i) implies (iii). Assume that [a] = [b]. It follows that 
[a]  [b] 4 B because [a] is nonempty (because a € [a] because R is reflexive). 

Next, we will show that (iii) implies (7). Suppose that [a] M[b] 4 0. Then there is an 
element c with c € [a] and c € [b]. In other words, aRc and bRc. By the symmetric 
property, cRb. Then by transitivity, because aRc and cRb, we have aRb. 

Because (7) implies (i), (ii) implies (di7), and (iii) implies (i), the three statements, (i), (ii), 
and (iii), are equivalent. 


We are now in a position to show how an equivalence relation partitions a set. Let R be an 
equivalence relation on a set A. The union of the equivalence classes of R is all of A, because 
an element a of A is in its own equivalence class, namely, [a]. In other words, 


Ulale = 4. 
acA 


In addition, from Theorem 1, it follows that these equivalence classes are either equal or disjoint, 
so 


[alr [blr = 9, 


when [ale # [D]r. 

These two observations show that the equivalence classes form a partition of A, because 
they split A into disjoint subsets. More precisely, a partition of a set S is a collection of disjoint 
nonempty subsets of § that have S as their union. In other words, the collection of subsets A;, 
i € I (where / is an index set) forms a partition of S if and only if 


A; #9% fori € J, 


Aj 1 Aj =@ wheni F j, 


EXAMPLE 12 


THEOREM 2 


EXAMPLE 13 
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Lad 


FIGURE 1 A Partition of a Set. 


and 


[Jai=S. 


ie] 


(Here the notation |_);.; Aj represents the union of the sets A; for alli € /.) Figure 1 illustrates 
the concept of a partition of a set. 


Suppose that S = {1,2,3,4,5,6}. The collection of sets A; = {1, 2,3}, Ao = {4,5}, and 
A3 = {6} forms a partition of S, because these sets are disjoint and their union is S. < 


We have seen that the equivalence classes of an equivalence relation on a set form a partition 
of the set. The subsets in this partition are the equivalence classes. Conversely, every partition 
of a set can be used to form an equivalence relation. Two elements are equivalent with respect 
to this relation if and only if they are in the same subset of the partition. 

To see this, assume that {A; | i € 7} is a partition on S. Let R be the relation on S consisting 
of the pairs (x, y), where x and y belong to the same subset A; in the partition. To show that R 
is an equivalence relation we must show that R is reflexive, symmetric, and transitive. 

We see that (a, a) € R for every a € S, because a is in the same subset as itself. Hence, R 
is reflexive. If (a, b) € R, then b and a are in the same subset of the partition, so that (b, a) € R 
as well. Hence, R is symmetric. If (a,b) € R and (b,c) € R, then a and D are in the same 
subset X in the partition, and b and c are in the same subset Y of the partition. Because the subsets 
of the partition are disjoint and b belongs to X and Y, it follows that X = Y. Consequently, a 
and c belong to the same subset of the partition, so (a, c) € R. Thus, R is transitive. 

It follows that R is an equivalence relation. The equivalence classes of R consist of subsets 
of S containing related elements, and by the definition of R, these are the subsets of the partition. 
Theorem 2 summarizes the connections we have established between equivalence relations and 
partitions. 


Let R be an equivalence relation on a set S. Then the equivalence classes of R form a partition 
of S. Conversely, given a partition {A; | i € 7} of the set S, there is an equivalence relation 
R that has the sets A;, i € J, as its equivalence classes. 


Example 13 shows how to construct an equivalence relation from a partition. 


List the ordered pairs in the equivalence relation R produced by the partition A; = {1, 2, 3}, 
A2 = {4, 5}, and A3 = {6} of S = {1, 2, 3, 4,5, 6}, given in Example 12. 
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EXAMPLE 14 


EXAMPLE 15 


Solution: The subsets in the partition are the equivalence classes of R. The pair (a, b) € Rif and 
only if a and b are in the same subset of the partition. The pairs (1, 1), (1, 2), , 3), (2, 1), (2, 2), 
(2, 3), (3, 1), G, 2), and (3, 3) belong to R because A; = {1, 2, 3} is an equivalence class; the 
pairs (4, 4), (4, 5), (5, 4), and (5, 5) belong to R because Az = {4, 5} is an equivalence class; 
and finally the pair (6, 6) belongs to R because {6} is an equivalence class. No pair other than 
those listed belongs to R. 


The congruence classes modulo m provide a useful illustration of Theorem 2. There 
are m different congruence classes modulo m, corresponding to the m different remainders 
possible when an integer is divided by m. These m congruence classes are denoted by 
[O]m, [1]m,---, [#7 — 1m. They form a partition of the set of integers. 


What are the sets in the partition of the integers arising from congruence modulo 4? 
Solution: There are four congruence classes, corresponding to [O]4, [1]4, [2]4, and [3]4. They 
are the sets 


(Ol 48) $40,485 
Sis ee a ae 2 
(2h =o, S629, 2-6. 10... 
Shah a5, <1, 3 alist 


These congruence classes are disjoint, and every integer is in exactly one of them. In other 
words, as Theorem 2 says, these congruence classes form a partition. 


We now provide an example of a partition of the set of all strings arising from an equivalence 
relation on this set. 


Let R3 be the relation from Example 5. What are the sets in the partition of the set of all bit strings 
arising from the relation R3 on the set of all bit strings? (Recall that s R3t, where s and t are bit 
strings, if s = ¢ or s and ¢ are bit strings with at least three bits that agree in their first three bits.) 
Solution: Note that every bit string of length less than three is equivalent only to itself. 
Hence [A]e, = {4}, [0] ay = {0}, [1]e; = (1},[00]a, = {00}, (O1]e, = {01}, 10] x, = {10}, and 
[11]r, = {11}. Note that every bit string of length three or more is equivalent to one of the eight 
bit strings 000, 001, 010, 011, 100, 101, 110, and 111. We have 


[000] rx; = {000, 0000, 0001, 00000, 00001, 00010, 00011, ...}, 
[O01]r,; = {001, 0010, 0011, 00100, 00101, 00110, 00111, ...}, 
[010] rx, = {010, 0100, 0101, 01000, 01001, 01010, 01011, ...}, 
[Oll]r, = {011, 0110, 0111, 01100, 01101, 01110, O1111,...}, 
[100] r, = {100, 1000, 1001, 10000, 10001, 10010, 10011, ...}, 
[101] r, = {101, 1010, 1011, 10100, 10101, 10110, 10111, ...}, 
[110]r, = {110, 1100, 1101, 11000, 11001, 11010, 11011, ...}, 


[111 ]r, = {111, 1110, 1111, 11100, 11101, 11110, 11111,...}. 


These 15 equivalence classes are disjoint and every bit string is in exactly one of them. As 
Theorem 2 tells us, these equivalence classes partition the set of all bit strings. < 


Exercises 


10. 


. Which of these relations on {0, 1, 2,3} are equivalence 


relations? Determine the properties of an equivalence re- 
lation that the others lack. 
a) {(0, 0), (1, 1), (2, 2), 3, 3)} 
b) {(0, 0), ©, 2), (2, 0), (2, 2), (2, 3), 3, 2), 3, 3)} 
ce) {(0, 0), (1, 1), C1, 2), 2, 1), 2, 2), 3, 3)} 
d) {(0, 0), (1, 1), C3), 2, 2), (2, 3), 3, 1), G, 2), 
(3, 3)} 
e) {(0,0), (0, 1), (0, 2), 1, 0), Gd, 1), , 2), (2, 0), 
(2, 2), (3, 3)} 


. Which of these relations on the set of all people are equiv- 


alence relations? Determine the properties of an equiva- 
lence relation that the others lack. 

a) {(a, b) | a and b are the same age} 

b) {(a, b) | a and b have the same parents} 

c) {(a, b) | a and b share a common parent} 

d) {(a, b) | a and b have met} 

e) {(a, b) | a and b speak a common language} 


. Which of these relations on the set of all functions from Z 


to Z are equivalence relations? Determine the properties 

of an equivalence relation that the others lack. 

a) {((f, 8) | fC) = gD} 

b) {(f, 8) | FO) = gO) or fC) = g()} 

ce) {(f,8) | f@) — g(x) = I forall x € Z} 

d) {(f,g)| for some C € Z, for all x € Z, f(x) — 
g(x) = C} 

e) {(f, 8) | fF) = gb and f(1) = g(0)} 


. Define three equivalence relations on the set of students 


in your discrete mathematics class different from the re- 
lations discussed in the text. Determine the equivalence 
classes for each of these equivalence relations. 


. Define three equivalence relations on the set of buildings 


on a college campus. Determine the equivalence classes 
for each of these equivalence relations. 


. Define three equivalence relations on the set of classes of- 


fered at your school. Determine the equivalence classes 
for each of these equivalence relations. 


. Show that the relation of logical equivalence on the set 


of all compound propositions is an equivalence relation. 
What are the equivalence classes of F and of T? 


. Let R be the relation on the set of all sets of real numbers 


such that S RT if and only if S and T have the same 
cardinality. Show that R is an equivalence relation. What 
are the equivalence classes of the sets {0, 1, 2} and Z? 


. Suppose that A is anonempty set, and f is a function that 


has A as its domain. Let R be the relation on A consisting 
of all ordered pairs (x, y) such that f(x) = f(y). 

a) Show that R is an equivalence relation on A. 

b) What are the equivalence classes of R? 

Suppose that A is a nonempty set and R is an equivalence 
relation on A. Show that there is a function f with A as its 
domain such that (x, y) € R if and only if f(x) = f(y). 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 
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Show that the relation R consisting of all pairs (x, y) such 
that x and y are bit strings of length three or more that 
agree in their first three bits is an equivalence relation on 
the set of all bit strings of length three or more. 


Show that the relation R consisting of all pairs (x, y) such 
that x and y are bit strings of length three or more that 
agree except perhaps in their first three bits is an equiva- 
lence relation on the set of all bit strings of length three 
or more. 


Show that the relation R consisting of all pairs (x, y) such 
that x and y are bit strings that agree in their first and third 
bits is an equivalence relation on the set of all bit strings 
of length three or more. 


Let R be the relation consisting of all pairs (x, y) such 
that x and y are strings of uppercase and lowercase En- 
glish letters with the property that for every positive in- 
teger n, the nth characters in x and y are the same letter, 
either uppercase or lowercase. Show that R is an equiva- 
lence relation. 


Let R be the relation on the set of ordered pairs of posi- 
tive integers such that ((a, b), (c, d)) € R if and only if 
a+dz=b-+c. Show that R is an equivalence relation. 


Let R be the relation on the set of ordered pairs of posi- 
tive integers such that ((a, b), (c, d)) € R if and only if 
ad = bc. Show that R is an equivalence relation. 


(Requires calculus) 


a) Show that the relation R on the set of all differentiable 
functions from R to R consisting of all pairs (f, g) 
such that f’(x) = g(x) for all real numbers x is an 
equivalence relation. 


b) Which functions are in the same equivalence class as 
the function f (x) = x?? 
(Requires calculus) 


a) Let n be a positive integer. Show that the relation 
R on the set of all polynomials with real-valued 
coefficients consisting of all pairs (f, g) such that 
f™ (x) = g(x) is an equivalence relation. [Here 
f(x) is the nth derivative of f (x).] 


b) Which functions are in the same equivalence class as 
the function f(x) = x4, where n = 3? 

Let R be the relation on the set of all URLs (or Web ad- 

dresses) such that x R y if and only if the Web page at 

x is the same as the Web page at y. Show that R is an 

equivalence relation. 


Let R be the relation on the set of all people who have 
visited a particular Web page such that x R y if and only 
if person x and person y have followed the same set of 
links starting at this Web page (going from Web page to 
Web page until they stop using the Web). Show that R is 
an equivalence relation. 
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In Exercises 21—23 determine whether the relation with the 
directed graph shown is an equivalence relation. 


21. 


23. 


24, 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


22. 


a 
is) 


d c 


Determine whether the relations represented by these 
zero—one matrices are equivalence relations. 


i Oi @ 
010 
io 4 ®) 


See 


1 1 
11 
11 


ooo 


111 
aoa el) peak es 
tid ea 
0101 0001 
Show that the relation R on the set of all bit strings such 
that s Rt if and only if s and ¢ contain the same number 
of 1s is an equivalence relation. 


What are the equivalence classes of the equivalence rela- 
tions in Exercise 1? 


What are the equivalence classes of the equivalence rela- 
tions in Exercise 2? 


What are the equivalence classes of the equivalence rela- 
tions in Exercise 3? 


What is the equivalence class of the bit string 011 for the 
equivalence relation in Exercise 25? 


What are the equivalence classes of these bit strings for 
the equivalence relation in Exercise 11? 


a) 010 b) 1011 c) 11111 d) 01010101 


What are the equivalence classes of the bit strings 
in Exercise 30 for the equivalence relation from 
Exercise 12? 


What are the equivalence classes of the bit strings 
in Exercise 30 for the equivalence relation from 
Exercise 13? 


What are the equivalence classes of the bit strings in Ex- 
ercise 30 for the equivalence relation R4 from Exam- 
ple 5 on the set of all bit strings? (Recall that bit strings s 
and t¢ are equivalent under R4 if and only if they are equal 
or they are both at least four bits long and agree in their 
first four bits.) 


What are the equivalence classes of the bit strings in Ex- 
ercise 30 for the equivalence relation Rs from Example 5 
on the set of all bit strings? (Recall that bit strings s and 
t are equivalent under Rs if and only if they are equal or 
they are both at least five bits long and agree in their first 
five bits.) 


35. 


36. 


37. 


38. 


39. 


40. 


41. 


42. 


43. 


44, 


What is the congruence class [n]5 (that is, the equiva- 
lence class of n with respect to congruence modulo 5) 
when n is 


a) 2? b) 3? c) 6? d) —3? 
What is the congruence class [4],,, when m is 

a) 2? b) 3? c) 6? d) 8? 

Give a description of each of the congruence classes 
modulo 6. 


What is the equivalence class of each of these strings with 

respect to the equivalence relation in Exercise 14? 

a) No b) Yes c) Help 

a) What is the equivalence class of (1, 2) with respect to 
the equivalence relation in Exercise 15? 

b) Give an interpretation of the equivalence classes for 
the equivalence relation R in Exercise 15. [Hint: Look 
at the difference a — b corresponding to (a, b).] 

a) What is the equivalence class of (1, 2) with respect 
to the equivalence relation in Exercise 16? 

b) Give an interpretation of the equivalence classes for 
the equivalence relation R in Exercise 16. [Hint: Look 
at the ratio a/b corresponding to (a, b).] 

Which of these collections of subsets are partitions of 

{1, 2,3, 4,5, 6}? 

a) {1, 2}, {2, 3, 4}, {4,5,6} b) {1}, (2, 3, 6}, {4}, {5} 

c) {2,4, 6}, {1, 3, 5} d) {1, 4, 5}, {2, 6} 

Which of these collections of subsets are partitions of 

{—3, —2, —1, 0, 1, 2, 3}? 

a) {—3, -1, 1, 3}, {—2, 0, 2} 

b) {—3, —2, —1, 0}, {0, 1, 2,3 

c) {-3, 3}, {-2, 2}, {-l, 1}, { 

d) {—3, —2, 2, 3}, {-1, 

Which of these collections of subsets are partitions of the 

set of bit strings of length 8? 


a) the set of bit strings that begin with 1, the set of bit 
strings that begin with 00, and the set of bit strings 
that begin with 01 

b) the set of bit strings that contain the string 00, the set 
of bit strings that contain the string 01, the set of bit 
strings that contain the string 10, and the set of bit 
strings that contain the string 11 

c) the set of bit strings that end with 00, the set of bit 
strings that end with 01, the set of bit strings that end 
with 10, and the set of bit strings that end with 11 

d) the set of bit strings that end with 111, the set of bit 
strings that end with 011, and the set of bit strings that 
end with 00 

e) the set of bit strings that contain 3k ones for some 
nonnegative integer k; the set of bit strings that con- 
tain 3k + 1 ones for some nonnegative integer k; and 
the set of bit strings that contain 3k + 2 ones for some 
nonnegative integer k. 

Which of these collections of subsets are partitions of the 

set of integers? 

a) the set of even integers and the set of odd integers 

b) the set of positive integers and the set of negative in- 
tegers 


} 
O} 


45. 


46. 


47. 


48. 


c) the set of integers divisible by 3, the set of integers 
leaving a remainder of 1 when divided by 3, and the 
set of integers leaving a remainder of 2 when divided 
by 3 

d) the set of integers less than —100, the set of integers 
with absolute value not exceeding 100, and the set of 
integers greater than 100 


e) the set of integers not divisible by 3, the set of even 
integers, and the set of integers that leave a remainder 
of 3 when divided by 6 

Which of these are partitions of the set Z x Z of ordered 

pairs of integers? 

a) the set of pairs (x, y), where x or y is odd; the set 
of pairs (x, y), where x is even; and the set of pairs 
(x, y), where y is even 


b) the set of pairs (x, y), where both x and y are odd; 
the set of pairs (x, y), where exactly one of x and y 
is odd; and the set of pairs (x, y), where both x and y 
are even 


c) the set of pairs (x, y), where x is positive; the set of 
pairs (x, y), where y is positive; and the set of pairs 
(x, y), where both x and y are negative 

d) the set of pairs (x, y), where 3 | x and 3 | y; the set 
of pairs (x, y), where 3 | x and3 / y; the set of pairs 
(x, y), where 3 / x and 3| y; and the set of pairs 
(x, y), where 3 7 x and3 / y 

e) the set of pairs (x, y), where x > 0 and y > 0; the 
set of pairs (x, y), where x > 0 and y < 0; the set of 
pairs (x, y), where x < 0 and y > 0; and the set of 
pairs (x, y), where x < Oand y <0 

f) the set of pairs (x, y), where x ~ Oand y ¥ 0; the set 
of pairs (x, y), where x = 0 and y ¥ 0; and the set of 
pairs (x, y), where x £4 O and y=0 

Which of these are partitions of the set of real numbers? 

a) the negative real numbers, {0}, the positive real 
numbers 

b) the set of irrational numbers, the set of rational 
numbers 


c) the set of intervals [k,k + 1],k =...,—2,-—1,0, 
15.2) ees 

d) the set of intervals (k,k +1),k =...,—2,—1,0, 
TDs ot 

e) the set of intervals (k,k + 1],k=...,—2,—1,0, 
Tg De ress 


f) the sets {x +n |n € Z} forall x € [0, 1) 

List the ordered pairs in the equivalence relations pro- 
duced by these partitions of {0, 1, 2, 3, 4, 5}. 

a) {0}, {1, 2}, (3,4, 5} 

b) {0, 1}, {2, 3}, {4, 5} 

c) {0, 1, 2}, (3, 4, 5} 

d) {O}, {1}, {2}, {3}, {4}, {5} 

List the ordered pairs in the equivalence relations pro- 
duced by these partitions of {a, b, c,d, e, f, g}. 


a) {a,b}, {c,d}, fe, f, g} 
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b) {a}, {b}, {c, d}, fe, f}, {3} 
c) {a,b,c, d}, fe, f, 3} 
d) {a,c,e, g}, {b, d}, {f} 


A partition P; is called a refinement of the partition P2 if 
every set in P, is a subset of one of the sets in Po. 


49. 


50. 


51. 


Show that the partition formed from congruence classes 
modulo 6 is a refinement of the partition formed from 
congruence classes modulo 3. 


Show that the partition of the set of people living in the 
United States consisting of subsets of people living in the 
same county (or parish) and same state is a refinement of 
the partition consisting of subsets of people living in the 
same state. 


Show that the partition of the set of bit strings of length 16 
formed by equivalence classes of bit strings that agree on 
the last eight bits is a refinement of the partition formed 
from the equivalence classes of bit strings that agree on 
the last four bits. 


In Exercises 52 and 53, R, refers to the family of equivalence 
relations defined in Example 5. Recall that s R, t, where s 
and ¢ are two strings if s = ¢ or s and ¢ are strings with at least 
n characters that agree in their first n characters. 


52. 


53. 


54, 


55. 


56. 


57. 


Show that the partition of the set of all bit strings formed 
by equivalence classes of bit strings with respect to the 
equivalence relation R,4 is a refinement of the partition 
formed by equivalence classes of bit strings with respect 
to the equivalence relation R3. 


Show that the partition of the set of all identifiers in C 
formed by the equivalence classes of identifiers with re- 
spect to the equivalence relation R3; is a refinement of 
the partition formed by equivalence classes of identifiers 
with respect to the equivalence relation Rg. (Compilers 
for “old” C consider identifiers the same when their names 
agree in their first eight characters, while compilers in 
standard C consider identifiers the same when their names 
agree in their first 31 characters.) 


Suppose that R; and R>2 are equivalence relations on a 
set A. Let P; and P> be the partitions that correspond to 
R, and Ro, respectively. Show that R; C Ro if and only 
if P; is arefinement of P). 


Find the smallest equivalence relation on the set 
{a,b,c,d,e} containing the relation {(a, b), (a,c), 


(d, e)}. 

Suppose that Rj and R2 are equivalence relations on the 

set S. Determine whether each of these combinations 

of R; and R2 must be an equivalence relation. 

a) Ri UR> b) Ri N Ro c) Ri @ Ro 

Consider the equivalence relation from Example 2, 

namely, R = {(x, y) | x — y is an integer}. 

a) What is the equivalence class of | for this equivalence 
relation? 


b) What is the equivalence class of 1/2 for this equiva- 
lence relation? 


618 9/ Relations 


*58. Each bead on a bracelet with three beads is either red, 61. Determine the number of different equivalence relations 
white, or blue, as illustrated in the figure shown. on a set with three elements by listing them. 


62. Determine the number of different equivalence relations 


Bead 1 ‘ Mee 
se on a set with four elements by listing them. 


Red 
* 63. Do we necessarily get an equivalence relation when we 


form the transitive closure of the symmetric closure of 
the reflexive closure of a relation? 


Bead3 Beaa2 * 64. Do we necessarily get an equivalence relation when we 
Blue White form the symmetric closure of the reflexive closure of the 


. transitive closure of a relation? 
Define the relation R between bracelets as: (B,, Bo), 


where B, and Bp are bracelets, belongs to R if and only 
if By can be obtained from B, by rotating it or rotating it 
and then reflecting it. 


65. Suppose we use Theorem 2 to form a partition P from 
an equivalence relation R. What is the equivalence rela- 
tion R’ that results if we use Theorem 2 again to form an 


. ; ; equivalence relation from P? 
a) Show that R is an equivalence relation. : 
66. Suppose we use Theorem 2 to form an equivalence rela- 


b) What are the equivalence classes of R? tion R from a partition P. What is the partition P’ that 


* 59. Let R be the relation on the set of all colorings of the 2 x 2 results if we use Theorem 2 again to form a partition 
checkerboard where each of the four squares is colored from R? 
either red or blue so that (Ci, C2), where Cy and C2 are 67. Devise an algorithm to find the smallest equivalence re- 


2 x 2 checkerboards with each of their four squares col- 
ored blue or red, belongs to R if and only if C2 can be 
obtained from C| either by rotating the checkerboard or 
by rotating it and then reflecting it. 


lation containing a given relation. 


*68. Let p(n) denote the number of different equivalence 
relations on a set with n elements (and by Theo- 
rem 2 the number of partitions of a set with n ele- 


a) Show that R is an equivalence relation. ments). Show that p(n) satisfies the recurrence relation 


b) What are the equivalence classes of R? p(n) = ae C(n —1, j)p(n — j — 1) and the initial 
60. a) Let R be the relation on the set of functions from Zt condition p(0) = 1. (Note: The numbers p(n) are called 
to Z* such that (f, g) belongs to R if and only if f Bell numbers after the American mathematician E. T. 
is O(g) (see Section 3.2). Show that R is an equiva- Bell.) 
lence relation. 69. Use Exercise 68 to find the number of different equiv- 


b) Describe the equivalence class containing f(n) =n? alence relations on a set with n elements, where n is a 


for the equivalence relation of part (a). positive integer not exceeding 10. 


9.6 Partial Orderings 


Introduction 


We often use relations to order some or all of the elements of sets. For instance, we order words 
using the relation containing pairs of words (x, y), where x comes before y in the dictionary. 
We schedule projects using the relation consisting of pairs (x, y), where x and y are tasks in 
a project such that x must be completed before y begins. We order the set of integers using 
the relation containing the pairs (x, y), where x is less than y. When we add all of the pairs 
= of the form (x, x) to these relations, we obtain a relation that is reflexive, antisymmetric, and 
Links transitive. These are properties that characterize relations used to order the elements of sets. 


DEFINITION 1  Arelation R ona set S is called a partial ordering or partial order if it is reflexive, antisym- 
metric, and transitive. A set S together with a partial ordering R is called a partially ordered 
set, or poset, and is denoted by (S, R). Members of S are called elements of the poset. 


We give examples of posets in Examples 1-3. 


EXAMPLE 1 — Show that the “greater than or equal” relation (>) is a partial ordering on the set of integers. 


Extra pe 
Examples “e 


EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 
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DEFINITION 2 


EXAMPLE 5 


DEFINITION 3 
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Solution: Because a > a for every integer a, > is reflexive. If a > b and b >a, thena = b. 
Hence, > is antisymmetric. Finally, > is transitive because a > b and b > c imply that a > c. 


It follows that > is a partial ordering on the set of integers and (Z, >) is a poset. 


The divisibility relation | is a partial ordering on the set of positive integers, because it is 
reflexive, antisymmetric, and transitive, as was shown in Section 9.1. We see that (Z*, |) is a 
poset. Recall that (Z* denotes the set of positive integers.) 


Show that the inclusion relation C is a partial ordering on the power set of a set S. 


Solution: Because A C A whenever A isa subset of S, C is reflexive. It is antisymmetric because 
A C Band B C Aimply that A = B. Finally, C is transitive, because A C B and B C C imply 
that A C C. Hence, C is a partial ordering on P(S), and (P(S), C) is a poset. < 


Example 4 illustrates a relation that is not a partial ordering. 


Let R be the relation on the set of people such that xR y if x and y are people and x is older 
than y. Show that R is not a partial ordering. 


Solution: Note that R is antisymmetric because if a person x is older than a person y, then y 
is not older than x. That is, if xR y, then y Rx. The relation R is transitive because if person x 
is older than person y and y is older than person z, then x is older than z. That is, if xR y 
and yRz, then xR z. However, R is not reflexive, because no person is older than himself or 
herself. That is, x R x for all people x. It follows that R is not a partial ordering. 


In different posets different symbols such as <, C, and |, are used for a partial ordering. 
However, we need a symbol that we can use when we discuss the ordering relation in an 
arbitrary poset. Customarily, the notation a = b is used to denote that (a, b) € R in an arbitrary 
poset (S$, R). This notation is used because the “less than or equal to” relation on the set of real 
numbers is the most familiar example of a partial ordering and the symbol ~ is similar to the < 
symbol. (Note that the symbol = is used to denote the relation in any poset, not just the “less 
than or equals” relation.) The notation a < b denotes that a = b, but a 4 b. Also, we say “a is 
less than b” or “b is greater than a” if a < b. 

When a and b are elements of the poset (S, =<), it is not necessary that either a =< b 
or b = a. For instance, in (P(Z), C), {1, 2} is not related to {1,3}, and vice versa, because 
neither set is contained within the other. Similarly, in (Zt, |), 2 is not related to 3 and 3 is not 
related to 2, because 2 { 3 and3 J 2. This leads to Definition 2. 


The elements a and b of a poset (S, =<) are called comparable if eithera =< borb =< a. When 
a and b are elements of S such that neither a < bnorb = a,aand bare called incomparable. 


In the poset (Z*, |), are the integers 3 and 9 comparable? Are 5 and 7 comparable? 


Solution: The integers 3 and 9 are comparable, because 3 | 9. The integers 5 and 7 are incom- 
parable, because 5 J 7 and7 { 5. < 


The adjective “partial” is used to describe partial orderings because pairs of elements may 
be incomparable. When every two elements in the set are comparable, the relation is called a 
total ordering. 


If (S, =) is a poset and every two elements of S are comparable, S is called a totally ordered 
or linearly ordered set, and = 1s called a total order or a linear order. A totally ordered set 
is also called a chain. 
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EXAMPLE 6 


EXAMPLE 7 


DEFINITION 4 


EXAMPLE 8 


THEOREM 1 


The poset (Z, <) is totally ordered, because a < b or b < a whenever a and b are integers. “4 


The poset (Z*, | ) is not totally ordered because it contains elements that are incomparable, such 
as 5 and 7. 


In Chapter 6 we noted that (Z*, <) is well-ordered, where < is the usual “less than or equal 
to” relation. We now define well-ordered sets. 


(S, =<) 1s a well-ordered set if it is a poset such that = is a total ordering and every nonempty 
subset of S has a least element. 


The set of ordered pairs of positive integers, Zt x Z*, with (ay, a2) = (by, bo) if ay < by, or if 
a, = by and az < bz (the lexicographic ordering), is a well-ordered set. The verification of this 
is left as Exercise 53. The set Z, with the usual < ordering, is not well-ordered because the set 
of negative integers, which is a subset of Z, has no least element. < 


At the end of Section 5.3 we showed how to use the principle of well-ordered induction 
(there called generalized induction) to prove results about a well-ordered set. We now state and 
prove that this proof technique is valid. 


THE PRINCIPLE OF WELL-ORDERED INDUCTION Suppose that S is a well- 
ordered set. Then P(x) is true for all x € S, if 


INDUCTIVE STEP: For every y € S, if P(x) is true for all x € S with x < y, then P(y) 
is true. 


Proof: Suppose it is not the case that P(x) is true for all x € S. Then there is an element 
y € S such that P(y) is false. Consequently, the set A = {x € S| P(x) is false} is nonempty. 
Because S' is well ordered, A has a least element a. By the choice of a as a least element of A, 
we know that P(x) is true for all x € S with x ~ a. This implies by the inductive step P(a) is 
true. This contradiction shows that P(x) must be true for all x € S. 


Remark: We do not need a basis step in a proof using the principle of well-ordered induction 
because if xo is the least element of a well ordered set, the inductive step tells us that P(x) 
is true. This follows because there are no elements x € S$ with x < x9, so we know (using a 
vacuous proof) that P(x) is true for all x € S with x ~ xo. 


The principle of well-ordered induction is a versatile technique for proving results about 
well-ordered sets. Even when it is possible to use mathematical induction for the set of positive 
integers to prove a theorem, it may be simpler to use the principle of well-ordered induction, as 
we saw in Examples 5 and 6 in Section 6.2, where we proved a result about the well-ordered 
set (N x N, =) where = is lexicographic ordering on N x N. 


Lexicographic Order 


The words in a dictionary are listed in alphabetic, or lexicographic, order, which is based on the 
ordering of the letters in the alphabet. This is a special case of an ordering of strings on a set 


EXAMPLE 9 
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constructed from a partial ordering on the set. We will show how this construction works in any 
poset. 

First, we will show how to construct a partial ordering on the Cartesian product of two 
posets, (Aj, <1) and (Ao, <2). The lexicographic ordering =< on A, x Ao is defined by 
specifying that one pair is less than a second pair if the first entry of the first pair is less than 
(in A;) the first entry of the second pair, or if the first entries are equal, but the second entry of 
this pair is less than (in Az) the second entry of the second pair. In other words, (a1, a2) is less 
than (51, b2), that is, 


(a1, 42) ~ (b1, b2), 


either if aj <, b, or if both aj = by and a2 <2 bo. 
We obtain a partial ordering = by adding equality to the ordering < on A; x Ao. The 
verification of this is left as an exercise. 


Determine whether (3, 5) ~ (4, 8), whether (3, 8) ~ (4,5), and whether (4, 9) ~ (4, 11) in the 
poset (Z x Z, =), where = is the lexicographic ordering constructed from the usual < relation 
on Z. 


Solution: Because 3 < 4, it follows that (3,5) ~ (4,8) and that (3, 8) ~ (4,5). We have 
(4,9) ~ (4, 11), because the first entries of (4, 9) and (4, 11) are the same but 9 < 11. < 


In Figure 1 the ordered pairs in Z*x Z* that are less than (3,4) are highlighted. 
A lexicographic ordering can be defined on the Cartesian product of n posets (Aj, = 1), 
(Az, <2),---,; (An, <n). Define the partial ordering < on A, x Az x --- X Ay by 


(a1, a2, .++54n) ~< (by, bo, +5 Dn) 


if ay <; Dy, or if there is an integer i > O such that a; = bj,...,a; = bj, and aj, <j41 Dj+1. 
In other words, one n-tuple is less than a second n-tuple if the entry of the first n-tuple in the 
first position where the two n-tuples disagree is less than the entry in that position in the second 
n-tuple. 


° e ° ° e--- 
a7) 27 837) 47) 67) ©7 (7,7) 
° ro ° e ° ° e--- 
(1,6) (2,6) (3,6) (4,6) (5,6) (6,6) (7,6) 

° e r ° e---: 
(1.5) @,5)° “GBy5) (45) (G35) (6:5) “C75) 


° e e e ° ° e---: 
a4) 24 64 44 64 ©4 (7,4) 


° e e e e ° e---: 
(1.3) (2,3) @,3) 43) 6,3) (6,3) (7,3) 


e e e e.--- 
(1,2) (2,2) (3,2) (4,2) (5,2) (6,2) (7,2) 


e e e e e ® @ -:: 
Gl) 2) 681) 4) 6) ©) @,) 


FIGURE 1 The Ordered Pairs Less Than (3, 4) in Lexicographic Order. 
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EXAMPLE 10 


EXAMPLE 11 


Note that (1, 2, 3,5) ~ (1, 2, 4, 3), because the entries in the first two positions of these 4-tuples 
agree, but in the third position the entry in the first 4-tuple, 3, is less than that in the second 
4-tuple, 4. (Here the ordering on 4-tuples is the lexicographic ordering that comes from the 
usual “less than or equals” relation on the set of integers.) < 


We can now define lexicographic ordering of strings. Consider the strings a,a2... a), and 
bi bz... by, ona partially ordered set S. Suppose these strings are not equal. Let f be the minimum 
of m and n. The definition of lexicographic ordering is that the string aja2...dm is less than 
b,b2...b, if and only if 


(a, aQ,.+-, ar) a (b1, bo,..., b;), or 
(a1, 42,..., a) = (by, bo,..., by) andm <n, 


where ~< in this inequality represents the lexicographic ordering of S’. In other words, to de- 
termine the ordering of two different strings, the longer string is truncated to the length of the 
shorter string, namely, to t = min(m, n) terms. Then the t-tuples made up of the first ¢ terms of 
each string are compared using the lexicographic ordering on S’. One string is less than another 
string if the f-tuple corresponding to the first string is less than the t-tuple of the second string, 
or if these two f-tuples are the same, but the second string is longer. The verification that this is 
a partial ordering is left as Exercise 38 for the reader. 


Consider the set of strings of lowercase English letters. Using the ordering of letters in the 
alphabet, a lexicographic ordering on the set of strings can be constructed. A string is less than 
a second string if the letter in the first string in the first position where the strings differ comes 
before the letter in the second string in this position, or if the first string and the second string 
agree in all positions, but the second string has more letters. This ordering is the same as that 
used in dictionaries. For example, 


discreet < discrete, 

because these strings differ first in the seventh position, and e < f. Also, 
discreet < discreetness, 

because the first eight letters agree, but the second string is longer. Furthermore, 
discrete < discretion, 

because 


discrete < discreti. 


Hasse Diagrams 


Many edges in the directed graph for a finite poset do not have to be shown because they must be 
present. For instance, consider the directed graph for the partial ordering {(a, b) | a < b} on the 
set {1, 2,3, 4}, shown in Figure 2(a). Because this relation is a partial ordering, it is reflexive, 
and its directed graph has loops at all vertices. Consequently, we do not have to show these loops 
because they must be present; in Figure 2(b) loops are not shown. Because a partial ordering is 
transitive, we do not have to show those edges that must be present because of transitivity. For 
example, in Figure 2(c) the edges (1, 3), (1, 4), and (2, 4) are not shown because they must be 
present. If we assume that all edges are pointed “upward” (as they are drawn in the figure), we 
do not have to show the directions of the edges; Figure 2(c) does not show directions. 

In general, we can represent a finite poset (S, <) using this procedure: Start with the 
directed graph for this relation. Because a partial ordering is reflexive, a loop (a, a) is present at 
every vertex a. Remove these loops. Next, remove all edges that must be in the partial ordering 
because of the presence of other edges and transitivity. That is, remove all edges (x, y) for 
which there is an element z € S such that x ~ z and z < x. Finally, arrange each edge so that 


EXAMPLE 12 


EXAMPLE 13 


Links 
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1 1 1 
(a) (b) (c) 


FIGURE 2 Constructing the Hasse Diagram 
for ({1, 2, 3, 4}, <). 


its initial vertex is below its terminal vertex (as it is drawn on paper). Remove all the arrows on 
the directed edges, because all edges point “upward” toward their terminal vertex. 

These steps are well defined, and only a finite number of steps need to be carried out for 
a finite poset. When all the steps have been taken, the resulting diagram contains sufficient 
information to find the partial ordering, as we will explain later. The resulting diagram is called 
the Hasse diagram of (S, =< ), named after the twentieth-century German mathematician Helmut 
Hasse who made extensive use of them. 

Let (S, x) be a poset. We say that an element y € S covers an element x € S if x ~ y 
and there is no element z € S such that x <x z x y. The set of pairs (x, y) such that y 
covers x is called the covering relation of (S, =<). From the description of the Hasse diagram of 
a poset, we see that the edges in the Hasse diagram of (S, = ) are upwardly pointing edges cor- 
responding to the pairs in the covering relation of (S, =). Furthermore, we can recover a poset 
from its covering relation, because it is the reflexive transitive closure of its covering relation. 
(Exercise 31 asks for a proof of this fact.) This tells us that we can construct a partial ordering 
from its Hasse diagram. 


Draw the Hasse diagram representing the partial ordering {(a,b)|a divides b} on 
{1, 2, 3,4, 6, 8, 12}. 


Solution: Begin with the digraph for this partial order, as shown in Figure 3(a). Remove all 
loops, as shown in Figure 3(b). Then delete all the edges implied by the transitive property. 
These are (1, 4), (1, 6), C1, 8), C1, 12), (2, 8), (2, 12), and (3, 12). Arrange all edges to point 
upward, and delete all arrows to obtain the Hasse diagram. The resulting Hasse diagram is shown 
in Figure 3(c). < 


Draw the Hasse diagram for the partial ordering {(A, B) | A C B} on the power set P(S) where 
S = {a,b,c}. 


HELMUT HASSE (1898-1979) Helmut Hasse was born in Kassel, Germany. He served in the German 
navy after high school. He began his university studies at Gottingen University in 1918, moving in 1920 to 
Marburg University to study under the number theorist Kurt Hensel. During this time, Hasse made fundamental 
contributions to algebraic number theory. He became Hensel’s successor at Marburg, later becoming director 
of the famous mathematical institute at Géttingen in 1934, and took a position at Hamburg University in 1950. 
Hasse served for 50 years as an editor of Crelle’s Journal, a famous German mathematics periodical, taking over 
the job of chief editor in 1936 when the Nazis forced Hensel to resign. During World War II Hasse worked on 
applied mathematics research for the German navy. He was noted for the clarity and personal style of his lectures 
and was devoted both to number theory and to his students. (Hasse has been controversial for connections with 
the Nazi party. Investigations have shown he was a strong German nationalist but not an ardent Nazi.) 
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EXAMPLE 14 


8@ 12 
4 6 
2 3 


(c) 
FIGURE 3 _ Constructing the Hasse Diagram of ({1, 2, 3, 4, 6, 8, 12}, |). 


Solution: The Hasse diagram for this partial ordering is obtained from the associated digraph by 
deleting all the loops and all the edges that occur from transitivity, namely, (4, {a, b}), (G, {a, c}), 
(G, {b, c}), GY, {a, b, c}), Cla}, {a, b, c}), ({b}, {a, b, c}), and ({c}, {a, b, c}). Finally all edges 
point upward, and arrows are deleted. The resulting Hasse diagram is illustrated in Figure 4. 4 


Maximal and Minimal Elements 


Elements of posets that have certain extremal properties are important for many applications. 
An element of a poset is called maximal if it is not less than any element of the poset. That is, a 
is maximal in the poset (S, =<) if there is no b € S such that a < b. Similarly, an element of a 
poset is called minimal if it is not greater than any element of the poset. That is, a is minimal 
if there is no element b € S such that b < a. Maximal and minimal elements are easy to spot 
using a Hasse diagram. They are the “top” and “bottom” elements in the diagram. 


Which elements of the poset ({2, 4, 5, 10, 12, 20, 25}, |) are maximal, and which are minimal? 


Solution: The Hasse diagram in Figure 5 for this poset shows that the maximal elements 
are 12, 20, and 25, and the minimal elements are 2 and 5. As this example shows, a poset 
can have more than one maximal element and more than one minimal element. < 


Sometimes there is an element in a poset that is greater than every other element. Such an 
element is called the greatest element. That is, a is the greatest element of the poset (S, =<) 


{a, b, c} 
12¢@ 20 
{a, c} {b, c} 
. < 4 10 25 
{a} {b} 
p 5 
FIGURE 4 _ The Hasse Diagram FIGURES The Hasse 


of (P ({a, b, c}), CS). Diagram of a Poset. 
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FIGURE7 The 
Hasse Diagram of 
a Poset. 
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a a b a b a 


(a) (b) (c) (d) 
FIGURE 6 Hasse Diagrams of Four Posets. 


if b x a for all b € S. The greatest element is unique when it exists [see Exercise 40(a)]. 
Likewise, an element is called the least element if it is less than all the other elements in the 
poset. That is, a is the least element of (S, =) ifa < bforallb € S. The least element is unique 
when it exists [see Exercise 40(b)]. 


Determine whether the posets represented by each of the Hasse diagrams in Figure 6 have a 
greatest element and a least element. 


Solution: The least element of the poset with Hasse diagram (a) is a. This poset has no greatest 
element. The poset with Hasse diagram (b) has neither a least nor a greatest element. The poset 
with Hasse diagram (c) has no least element. Its greatest element is d. The poset with Hasse 
diagram (d) has least element a and greatest element d. < 


Let S be a set. Determine whether there is a greatest element and a least element in the poset 
(P(S), S). 


Solution: The least element is the empty set, because @ C T for any subset T of S. The set S is 
the greatest element in this poset, because T C S whenever T is a subset of S. < 


Is there a greatest element and a least element in the poset (Z™, |)? 


Solution: The integer | is the least element because ||n whenever 7 is a positive integer. Because 
there is no integer that is divisible by all positive integers, there is no greatest element. < 


Sometimes it is possible to find an element that is greater than or equal to all the elements 
in a subset A of a poset (S, =<). If u is an element of S such that a = u for all elements a € A, 
then u is called an upper bound of A. Likewise, there may be an element less than or equal to 
all the elements in A. If / is an element of S such that / = a for all elements a € A, then / is 
called a lower bound of A. 


Find the lower and upper bounds of the subsets {a, b, c}, {j, h}, and {a, c,d, f} in the poset 
with the Hasse diagram shown in Figure 7. 


Solution: The upper bounds of {a, b, c} are e, f, 7, and h, and its only lower bound is a. There 
are no upper bounds of {j, 4}, and its lower bounds are a, b, c,d, e, and f. The upper bounds 
of {a,c,d, f} are f, h, and j, and its lower bound is a. 


The element x is called the least upper bound of the subset A if x is an upper bound that 
is less than every other upper bound of A. Because there is only one such element, if it exists, 
it makes sense to call this element the least upper bound [see Exercise 42(a)]. That is, x is the 
least upper bound of A if a = x whenever a € A, and x = z whenever z is an upper bound of 
A. Similarly, the element y is called the greatest lower bound of A if y is a lower bound of 
A and z = y whenever z is a lower bound of A. The greatest lower bound of A is unique if it 
exists [see Exercise 42(b)]. The greatest lower bound and least upper bound of a subset A are 
denoted by glb(A) and lub(A), respectively. 
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EXAMPLE 21 


Find the greatest lower bound and the least upper bound of {b, d, g}, if they exist, in the poset 
shown in Figure 7. 


Solution: The upper bounds of {b, d, g} are g and h. Because g < h, g is the least upper bound. 
The lower bounds of {b, d, g} are a and b. Because a < b, b is the greatest lower bound. < 


Find the greatest lower bound and the least upper bound of the sets {3, 9, 12} and {1, 2, 4, 5, 10}, 
if they exist, in the poset (Z*, |). 


Solution: An integer is a lower bound of {3, 9, 12} if 3, 9, and 12 are divisible by this integer. 
The only such integers are | and 3. Because | | 3, 3 is the greatest lower bound of {3, 9, 12}. 
The only lower bound for the set {1, 2, 4, 5, 10} with respect to | is the element 1. Hence, | is 
the greatest lower bound for {1, 2, 4, 5, 10}. 

An integer is an upper bound for {3, 9, 12} if and only if it is divisible by 3, 9, and 12. 
The integers with this property are those divisible by the least common multiple of 3, 9, and 
12, which is 36. Hence, 36 is the least upper bound of {3, 9, 12}. A positive integer is an upper 
bound for the set {1, 2, 4,5, 10} if and only if it is divisible by 1, 2, 4, 5, and 10. The integers 
with this property are those integers divisible by the least common multiple of these integers, 
which is 20. Hence, 20 is the least upper bound of {1, 2, 4, 5, 10}. < 


Lattices 


A partially ordered set in which every pair of elements has both a least upper bound and a 
greatest lower bound is called a lattice. Lattices have many special properties. Furthermore, 
lattices are used in many different applications such as models of information flow and play an 
important role in Boolean algebra. 


Determine whether the posets represented by each of the Hasse diagrams in Figure 8 are lattices. 


Solution: The posets represented by the Hasse diagrams in (a) and (c) are both lattices because 
in each poset every pair of elements has both a least upper bound and a greatest lower bound, 
as the reader should verify. On the other hand, the poset with the Hasse diagram shown in (b) 
is not a lattice, because the elements b and c have no least upper bound. To see this, note that 
each of the elements d, e, and f is an upper bound, but none of these three elements precedes 
the other two with respect to the ordering of this poset. 


a a 
(a) (b) (c) 


FIGURE 8 Hasse Diagrams of Three Posets. 


EXAMPLE 22 


EXAMPLE 23 


EXAMPLE 24 


EXAMPLE 25 


Links 


There are billions of 
pages of classified U.S. 
government documents. 
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Is the poset (Z*, |) a lattice? 


Solution: Let a and b be two positive integers. The least upper bound and greatest lower bound 
of these two integers are the least common multiple and the greatest common divisor of these 
integers, respectively, as the reader should verify. It follows that this poset is a lattice. < 


Determine whether the posets ({1, 2, 3, 4, 5}, |) and ({1, 2, 4, 8, 16}, |) are lattices. 


Solution: Because 2 and 3 have no upper bounds in ({1, 2, 3, 4, 5}, |), they certainly do not have 
a least upper bound. Hence, the first poset is not a lattice. 

Every two elements of the second poset have both a least upper bound and a greatest lower 
bound. The least upper bound of two elements in this poset is the larger of the elements and the 
greatest lower bound of two elements is the smaller of the elements, as the reader should verify. 
Hence, this second poset is a lattice. 


Determine whether (P(S), C) is a lattice where S is a set. 


Solution: Let A and B be two subsets of S. The least upper bound and the greatest lower bound 
of A and B are AU B and AN B, respectively, as the reader can show. Hence, (P(S), C) isa 
lattice. < 


The Lattice Model of Information Flow In many settings the flow of information from one 
person or computer program to another is restricted via security clearances. We can use a lattice 
model to represent different information flow policies. For example, one common information 
flow policy is the multilevel security policy used in government and military systems. Each 
piece of information is assigned to a security class, and each security class is represented by a 
pair (A, C) where A is an authority level and C is a category. People and computer programs 
are then allowed access to information from a specific restricted set of security classes. 

The typical authority levels used in the U.S. government are unclassified (0), confidential (1), 
secret (2), and top secret (3). (Information is said to be classified if it is confidential, secret, 
or top secret.) Categories used in security classes are the subsets of a set of all compartments 
relevant to a particular area of interest. Each compartment represents a particular subject area. 
For example, if the set of compartments is {spies, moles, double agents}, then there are eight 
different categories, one for each of the eight subsets of the set of compartments, such as 
{spies, moles}. 

We can order security classes by specifying that (A;, Cy) <(A2, C2) if and only if 
A, < A2 and C; C Co. Information is permitted to flow from security class (A, C1) into 
security class (Az, C2) if and only if (Aj, C,) <(A2, C2). For example, information is 
permitted to flow from the security class (secret, {spies, moles}) into the security class 
(top secret, {spies, moles, double agents}), whereas information is not allowed to flow from 
the security class (top secret, {spies, moles}) into either of the security classes (secret, {spies, 
moles, double agents}) or (top secret, {spies}). 

We leave it to the reader (see Exercise 48) to show that the set of all security classes with 
the ordering defined in this example forms a lattice. < 


Topological Sorting 


Suppose that a project is made up of 20 different tasks. Some tasks can be completed only after 
others have been finished. How can an order be found for these tasks? To model this problem 
we set up a partial order on the set of tasks so that a < b if and only if a and b are tasks where b 
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Links 


LEMMA 1 


EXAMPLE 26 


cannot be started until a has been completed. To produce a schedule for the project, we need to 
produce an order for all 20 tasks that is compatible with this partial order. We will show how 
this can be done. 

We begin with a definition. A total ordering =< is said to be compatible with the partial 
ordering R if a < b whenever aRb. Constructing a compatible total ordering from a partial 
ordering is called topological sorting.* We will need to use Lemma 1. 


Every finite nonempty poset (S, = ) has at least one minimal element. 


Proof: Choose an element ao of S. If ag is not minimal, then there is an element a, with a, < ao. 
If a, is not minimal, there is an element a2 with a2 ~ a;. Continue this process, so that if a, is 
not minimal, there is an element a, with a,41 < dy). Because there are only a finite number 
of elements in the poset, this process must end with a minimal element ay. | 


The topological sorting algorithm we will describe works for any finite nonempty poset. 
To define a total ordering on the poset (A, =), first choose a minimal element a;; such an 
element exists by Lemma 1. Next, note that (A — {a,}, =) is also a poset, as the reader should 
verify. (Here by < we mean the restriction of the original relation < on A to A — {aj}.) If itis 
nonempty, choose a minimal element az of this poset. Then remove az as well, and if there are 
additional elements left, choose a minimal element a3 in A — {a1, a2}. Continue this process by 
choosing a+) to be a minimal element in A — {aj, a2, ..., ax}, as long as elements remain. 

Because A is a finite set, this process must terminate. The end product is a sequence of 
elements a, 42, ..., @,. The desired total ordering =; is defined by 


Q\ <t G2 <t +++ <p Gy. 


This total ordering is compatible with the original partial ordering. To see this, note that if b ~ c 
in the original partial ordering, c is chosen as the minimal element at a phase of the algorithm 
where b has already been removed, for otherwise c would not be a minimal element. Pseudocode 
for this topological sorting algorithm is shown in Algorithm 1. 


ALGORITHM 1 Topological Sorting. 


procedure topological sort ((S, =<): finite poset) 


k:=1 

while S 4 
ay : = aminimal element of S {such an element exists by Lemma 1} 
S:= S— {ax} 
k:=k+1 

return |, 42,..., Qn {a,,d2,..., dy, is a compatible total ordering of S} 


Find a compatible total ordering for the poset ({1, 2, 4, 5, 12, 20}, |). 


*“Topological sorting” is terminology used by computer scientists; mathematicians use the terminology “linearization of a 
partial ordering” for the same thing. In mathematics, topology is the branch of geometry dealing with properties of geometric 
figures that hold for all figures that can be transformed into one another by continuous bijections. In computer science, a 
topology is any arrangement of objects that can be connected with edges. 
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A Cc E 


FIGURE 10 The 
Hasse Diagram for 
Seven Tasks. 
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12 20 12 20 12 20 | 12 20) 12 20 12 
be e ° e 
4 4 4 
4 
2 
5 2 5 2 
1 
Minimal 
element il 5 2 4 20 12 
chosen 


FIGURE 9 A Topological Sort of ({1, 2, 4, 5, 12, 20}, |). 


Solution: The first step is to choose a minimal element. This must be 1, because it is the only 
minimal element. Next, select a minimal element of ({2, 4, 5, 12, 20}, |). There are two minimal 
elements in this poset, namely, 2 and 5. We select 5. The remaining elements are {2, 4, 12, 20}. 
The only minimal element at this stage is 2. Next, 4 is chosen because it is the only minimal 
element of ({4, 12, 20}, |). Because both 12 and 20 are minimal elements of ({12, 20}, |), either 
can be chosen next. We select 20, which leaves 12 as the last element left. This produces the 
total ordering 


1~5 ~2 ~ 4 ~ 20 ~ 12. 
The steps used by this sorting algorithm are displayed in Figure 9. < 


Topological sorting has an application to the scheduling of projects. 


A development project at a computer company requires the completion of seven tasks. Some of 
these tasks can be started only after other tasks are finished. A partial ordering on tasks is set up 
by considering task X < task Y if task Y cannot be started until task X has been completed. The 
Hasse diagram for the seven tasks, with respect to this partial ordering, is shown in Figure 10. 
Find an order in which these tasks can be carried out to complete the project. 


Solution: An ordering of the seven tasks can be obtained by performing a topolog- 
ical sort. The steps of a sort are illustrated in Figure 11. The result of this sort, 


A ~<C ~< B < E < F < D <X G, gives one possible order for the tasks. < 
G G G G G G G 
D F D F D F | D F o/ Ne »/ 
B B B 
A Cc E c E E E 
Minimal 
element A Cc B E F D G 
chosen 


FIGURE 11 A Topological Sort of the Tasks. 
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Exercises 


. Which of these relations on {0, 1, 2, 3} are partial order- 
ings? Determine the properties of a partial ordering that 
the others lack. 


a) {(0,0), , D, @, 2), 3, 3)} 

b) {(0,0), C, I, @, 0), (2, 2), (2, 3), GB, 2), 3, 3)} 
ce) {(0,0), d, D, C, 2), 2,2), 3, 3)} 

d) {(0,0), CD, C, 2), C, 3), (2, 2), 2, 3), 3, 3)} 


e) {(0, 0), (0, 1), (0, 2), 1,0), C, 1), CL, 2), 2, 0), 

(2, 2), (3, 3)} 
. Which of these relations on {0, 1, 2, 3} are partial order- 
ings? Determine the properties of a partial ordering that 
the others lack. 


a) {(0, 0), (2, 2), (3, 3)} 
b) {(, 0), C, 1), @, 0), (2, 2), (2, 3), 3, 3)} 
ce) {(0,0), d, D, C, 2), 2, 2), B,D, GB, 3)} 


d) {(0, 0), C1, 1), (1, 2), (1, 3), @, 0), (2, 2), (2, 3), 
(3, 0), 3, 3)} 
e) {(0, 0), ©, 1), ©, 2), ©, 3), 1,0), d, D, , 2), 
(1, 3), (2, 0), (2, 2), (3, 3)} 
. Is (S, R) a poset if S is the set of all people in the world 
and (a, b) € R, where a and b are people, if 


a) ais taller than b? 
b) ais not taller than b? 
c) a=b ora isan ancestor of b? 


d) a and b have a common friend? 


. Is (S, R) a poset if S is the set of all people in the world 
and (a, b) € R, where a and b are people, if 
a) ais no shorter than b? 


b) a weighs more than b? 
c) a=bora isa descendant of b? 


d) a and b do not have a common friend? 
. Which of these are posets? 


a) (Z,=) b) (ZA) ©) (Z,2) dd (Z,/) 
. Which of these are posets? 
a) (R=) bR<) OC RX) @ RF) 


. Determine whether the relations represented by these 
zero—one matrices are partial orders. 


1 1 1 1 1 1 
a) }1 1 O b) |0 1 O 
0 0 1 0 0 1 
1 11 0 
y/o + 2-0 
001 1 
1 101 


8. 


Determine whether the relations represented by these 
zero—one matrices are partial orders. 


101 1 0 0 
a) }1 1 0 b) |O 1 O 
0 0 1 1 0 1 
101 0 
ne ee 
001 1 
110 1 


In Exercises 9-11 determine whether the relation with the 
directed graph shown is a partial order. 


9. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


10. 


OD 


Let (S$, R) be a poset. Show that (S, R7!) is also a poset, 
where R~! is the inverse of R. The poset (S, Ro!) is 
called the dual of (S, R). 

Find the duals of these posets. 

a) ({0, 1, 2}, <) b) (Z, >) 

c) (P(Z), 2) d) (Z*,|) 

Which of these pairs of elements are comparable in the 
poset (Zt, |)? 

a) 5,15 b) 6,9 c) 8,16 d) 7,7 
Find two incomparable elements in these posets. 

a) (P({0, 1, 2}), S) b) ({1, 2,4, 6, 8}, 1) 

Let S = {1, 2, 3, 4}. With respect to the lexicographic or- 
der based on the usual “less than” relation, 

a) find all pairs in S x S less than (2, 3). 

b) find all pairs in S x S greater than (3, 1). 

c) draw the Hasse diagram of the poset (S x S, =). 
Find the lexicographic ordering of these n-tuples: 

a) (1, 1,2), C1, 2, 1) b) (0, 1, 2,3), (0, 1, 3, 2) 
ce) (1,0, 1,0, 1), (0, 1, 1, 1, 0) 

Find the lexicographic ordering of these strings of lower- 
case English letters: 

a) quack, quick, quicksilver, quicksand, quacking 

b) open, opener, opera, operand, opened 

c) z00, zero, zoom, zoology, zoological 

Find the lexicographic ordering of the bit strings 0, 01, 
11, 001, 010, 011, 0001, and 0101 based on the ordering 
0<1. 


Draw the Hasse diagram for the “greater than or equal to” 
relation on {0, 1, 2, 3, 4, 5}. 


21 


22. 


23. 


24. 


. Draw the Hasse diagram for the “less than or equal to” 
relation on {0, 2,5, 10, 11, 15}. 

Draw the Hasse diagram for divisibility on the set 

a) {1,2,3,4,5, 6}. b) {3,5, 7, 11, 13, 16, 17}. 
ce) {2,3,5, 10,11, 15,25}. d) {1,3, 9, 27, 81, 243}. 
Draw the Hasse diagram for divisibility on the set 

a) {1,2,3,4,5,6,7, 8}. b) {1,2,3,5, 7, 11, 13}. 
c) {1, 2,3, 6, 12, 24, 36, 48}. 

d) {1, 2,4, 8, 16, 32, 64}. 

Draw the Hasse diagram for inclusion on the set P(S), 
where S = {a, b,c, d}. 


In Exercises 25—27 list all ordered pairs in the partial ordering 
with the accompanying Hasse diagram. 


25. 


27. 


28 


29 


30. 


31 


32 


26. 
c d e d 
b 
b € 
a 
a 
d e f 
& 
b a c 


. What is the covering relation of the partial ordering 
{(a, b) | a divides b} on {1, 2, 3, 4, 6, 12}? 


. What is the covering relation of the partial order- 
ing {(A, B) | AC B} on the power set of S, where 
S = {a,b,c}? 


. What is the covering relation of the partial ordering for 
the poset of security classes defined in Example 25? 


. Show that a finite poset can be reconstructed from its cov- 
ering relation. [Hint: Show that the poset is the reflexive 
transitive closure of its covering relation.] 


. Answer these questions for the partial order represented 
by this Hasse diagram. 


1 


a) Find the maximal elements. 
b) Find the minimal elements. 
c) Is there a greatest element? 


33. 


34. 


35. 


36. 


37. 


38. 


39. 
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d) Is there a least element? 

e) Find all upper bounds of {a, b, c}. 

f) Find the least upper bound of {a, D, c}, if it exists. 
g) Find all lower bounds of {f, g, h}. 


h) Find the greatest lower bound of { f, g, h}, if it exists. 


Answer these questions for the poset ({3,5,9, 15, 
24, 45}, |). 


a) Find the maximal elements. 

b) Find the minimal elements. 

c) Is there a greatest element? 

d) Is there a least element? 

e) Find all upper bounds of {3, 5}. 

f) Find the least upper bound of {3, 5}, if it exists. 

g) Find all lower bounds of {15, 45}. 

h) Find the greatest lower bound of {15, 45}, if it exists. 
Answer these questions for the poset ({2, 4, 6,9, 12, 
18, 27, 36, 48, 60, 72}, |). 

a) Find the maximal elements. 

b) Find the minimal elements. 

c) Is there a greatest element? 

d) Is there a least element? 

e) Find all upper bounds of {2, 9}. 

f) Find the least upper bound of {2, 9}, if it exists. 

g) Find all lower bounds of {60, 72}. 


h) Find the greatest lower bound of {60, 72}, if it exists. 
Answer these questions for the poset ({{1}, {2}, {4}, 
{1,2}, {1, 4}, (2, 4}, (3, 4}, {1, 3, 4}, {2, 3, 44}, ©). 

a) Find the maximal elements. 

b) Find the minimal elements. 

c) Is there a greatest element? 

d) Is there a least element? 

e) Find all upper bounds of {{2}, {4}}. 

f) Find the least upper bound of {{2}, {4}}, if it exists. 
g) Find all lower bounds of {{1, 3, 4}, {2, 3, 4}}. 


h) Find the greatest lower bound of {{1, 3, 4}, {2, 3, 4}}, 
if it exists. 

Give a poset that has 

a) a minimal element but no maximal element. 

b) a maximal element but no minimal element. 

c) neither a maximal nor a minimal element. 

Show that lexicographic order is a partial ordering on the 

Cartesian product of two posets. 


Show that lexicographic order is a partial ordering on the 
set of strings from a poset. 


Suppose that (S, <1) and (7, <2) are posets. Show that 
(S x T, =) is a poset where (s, t) = (u, v) if and only if 
§ <;uandt X2v. 
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40. 


41. 


42. 


43. 


44, 


45. 


46. 


47. 


48. 


a) Show that there is exactly one greatest element of a 
poset, if such an element exists. 

b) Show that there is exactly one least element of a poset, 
if such an element exists. 


a) Show that there is exactly one maximal element in a 
poset with a greatest element. 

b) Show that there is exactly one minimal element in a 
poset with a least element. 

a) Show that the least upper bound of a set in a poset is 
unique if it exists. 

b) Show that the greatest lower bound of a set in a poset 
is unique if it exists. 


Determine whether the posets with these Hasse diagrams 
are lattices. 


a) b) c) 
g h i 
7 f g h 
d : d . f g 
d e 
b c b é 
b c 
a 
a 
a 
Determine whether these posets are lattices. 
a) ({1, 3,6, 9, 12}, |) b) ({1, 5, 25, 125}, |) 
c) (Z, =) 


d) (P(S), 2), where P(S) is the power set of a set S 


Show that every nonempty finite subset of a lattice has a 
least upper bound and a greatest lower bound. 


Show that if the poset (S, R) is a lattice then the dual 
poset (S, R7!) is also a lattice. 


Inacompany, the lattice model of information flow is used 
to control sensitive information with security classes rep- 
resented by ordered pairs (A, C). Here A is an authority 
level, which may be nonproprietary (0), proprietary (1), 
restricted (2), or registered (3). A category C is a subset of 
the set of all projects {Cheetah, Impala, Puma}. (Names 
of animals are often used as code names for projects in 
companies.) 
a) Is information permitted to flow from (Proprietary, 
{Cheetah, Puma}) into (Restricted, {Puma})? 
b) Is information permitted to flow from (Restricted, 
{Cheetah}) into (Registered, {Cheetah, Impala})? 
c) Into which classes is information from (Proprietary, 
{Cheetah, Puma}) permitted to flow? 
d) From which classes is information permitted to flow 
into the security class (Restricted, {Impala, Puma})? 
Show that the set S of security classes (A, C) is a lattice, 
where A is a positive integer representing an authority 
class and C is a subset of a finite set of compartments, 
with (Aj, C}) =< (Az, C2) if and only if A; < Az and 
C, C Cp). [Hint: First show that (S$, =< ) is aposet and then 
show that the least upper bound and greatest lower bound 
of (Aj, C)) and (Az, C2) are (max(Aj, Az), Cy U C2) 
and (min(A;, Az), C, M C2), respectively.] 


* 49, Show that the set of all partitions of a set S with the re- 
lation P; = P> if the partition P; is a refinement of the 
partition P» is a lattice. (See the preamble to Exercise 49 
of Section 9.5.) 


50. Show that every totally ordered set is a lattice. 


51. Show that every finite lattice has a least element and a 
greatest element. 
52. Give an example of an infinite lattice with 
a) neither a least nor a greatest element. 
b) a least but not a greatest element. 
c) a greatest but not a least element. 
d) both a least and a greatest element. 
53. Verify that (Z* x Z*, <) isa well-ordered set, where < 
is lexicographic order, as claimed in Example 8. 
54. Determine whether each of these posets is well-ordered. 
a) (S, <), where S = {10, 11, 12,...} 
b) (QN (0, 1], <) (the set of rational numbers between 
0 and | inclusive) 
c) (S,<), where S is the set of positive rational numbers 
with denominators not exceeding 3 
d) (Z_, =>), where Z is the set of negative integers 
A poset (R, =) is well-founded if there is no infinite de- 
creasing sequence of elements in the poset, that is, elements 
X1,X2,...,Xn Such that --- ~ x, ~--+ ~ x2 ~ x1. A poset 


(R, =) is dense if for all x € S and y € S with x < y, there 
is an element z € R such that x ~ z < y. 


55. Show that the poset (Z, =), where x < y if and only if 
|x| < |y| is well-founded but is not a totally ordered set. 


56. Show that a dense poset with at least two elements that 
are comparable is not well-founded. 


57. Show that the poset of rational numbers with the usual 
“less than or equal to” relation, (Q, <), is a dense poset. 


*58. Show that the set of strings of lowercase English let- 
ters with lexicographic order is neither well-founded nor 
dense. 


59. Show that a poset is well-ordered if and only if it is totally 
ordered and well-founded. 


60. Show that a finite nonempty poset has a maximal element. 


61. Find a compatible total order for the poset with the Hasse 
diagram shown in Exercise 32. 


62. Find a compatible total order for the divisibility relation 
on the set {1, 2, 3, 6, 8, 12, 24, 36}. 


63. Find all compatible total orderings for the poset 
({1, 2, 4, 5, 12, 20}, |} from Example 26. 


64. Find all compatible total orderings for the poset with the 
Hasse diagram in Exercise 27. 


65. Find all possible orders for completing the tasks in the 
development project in Example 27. 


66. Schedule the tasks needed to build a house, by specifying 
their order, if the Hasse diagram representing these tasks 
is as shown in the figure. 


Completion 


Interior fixtures 
Exterior fixtures 


Carpeting Interior painting 


Flooring Exterior painting 
Wall-board 

Plumbing Wiring 
Exterior siding 


Roof 
Framing 


Foundation 
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TERMS 


binary relation from A to B: a subset of A x B 

relation on A: a binary relation from A to itself (i.e., a subset 
of A x A) 

So R: composite of R and S$ 

R™!: inverse relation of R 

R”: nth power of R 

reflexive: a relation R on A is reflexive if (a, a) € R for all 
aéeA 

symmetric: arelation R on A is symmetric if (b, a) € R when- 
ever (a,b) ER 

antisymmetric: a relation R on A is antisymmetric if a = b 
whenever (a,b) € Rand (b,a) ER 

transitive: a relation R on A is transitive if (a,b) € R and 
(b,c) € R implies that (a,c) € R 

n-ary relation on Aj, A2,..., An: a subset of A; x Az x 
ree, 4 An 

relational data model: a model for representing databases us- 
ing n-ary relations 

primary key: a domain of an n-ary relation such that an n- 
tuple is uniquely determined by its value for this domain 


composite key: the Cartesian product of domains of an n-ary 
relation such that an n-tuple is uniquely determined by its 
values in these domains 

selection operator: a function that selects the n-tuples in an 
n-ary relation that satisfy a specified condition 

projection: a function that produces relations of smaller de- 
gree from an n-ary relation by deleting fields 

join: a function that combines n-ary relations that agree on 
certain fields 

directed graph or digraph: a set of elements called vertices 
and ordered pairs of these elements, called edges 

loop: an edge of the form (a, a) 
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67. Find an ordering of the tasks of a software project if the 
Hasse diagram for the tasks of the project is as shown. 


Completion 


B test 
atest 


Develop module A Integrate modules 


Write Develop module B 


: \ 
documentation Develop module C 


Set up 


Develop system I 
test sites 


requirements 
Write functional requirements 


Determine user needs 


closure of a relation R with respect to a property P: the re- 
lation S (if it exists) that contains R, has property P, and 
is contained within any relation that contains R and has 
property P 

path in a digraph: a sequence of edges (a, x1), (%1, X2),.--, 
(Xn—2,; Xn—1), (Xn—1, D) such that the terminal vertex of each 
edge is the initial vertex of the succeeding edge in the se- 
quence 

circuit (or cycle) in a digraph: a path that begins and ends at 
the same vertex 

R* (connectivity relation): the relation consisting of those 
ordered pairs (a, b) such that there is a path from a to b 

equivalence relation: a reflexive, symmetric, and transitive 
relation 

equivalent: if R is an equivalence relation, a is equivalent to 
bifaRb 

[a]r (equivalence class of a with respect to R): the set of all 
elements of A that are equivalent to a 

[a]; (congruence class modulo m): the set of integers con- 
gruent to a modulo m 

partition of a set S: a collection of pairwise disjoint nonempty 
subsets that have S as their union 

partial ordering: a relation that is reflexive, antisymmetric, 
and transitive 

poset (S, R): aset S and a partial ordering R on this set 

comparable: the elements a and b in the poset (A, =) are 
comparable ifa x borb xa 

incomparable: elements in a poset that are not comparable 

total (or linear) ordering: a partial ordering for which every 
pair of elements are comparable 

totally (or linearly) ordered set: a poset with a total (or linear) 
ordering 

well-ordered set: a poset (S, =), where = is a total order 
and every nonempty subset of S has a least element 
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lexicographic order: a partial ordering of Cartesian products 
or strings 

Hasse diagram: a graphical representation of a poset where 
loops and all edges resulting from the transitive property 
are not shown, and the direction of the edges is indicated 
by the position of the vertices 

maximal element: an element of a poset that is not less than 
any other element of the poset 

minimal element: an element of a poset that is not greater than 
any other element of the poset 

greatest element: an element of a poset greater than all other 
elements in this set 

least element: an element of a poset less than all other elements 
in this set 

upper bound of a set: an element in a poset greater than all 
other elements in the set 

lower bound of a set: an element in a poset less than all other 
elements in the set 

least upper bound of a set: an upper bound of the set that is 
less than all other upper bounds 

greatest lower bound of a set: a lower bound of the set that 
is greater than all other lower bounds 

lattice: a partially ordered set in which every two elements 
have a greatest lower bound and a least upper bound 


Review Questions 


1. a) What is a relation on a set? 
b) How many relations are there on a set with n elements? 


2. a) What is a reflexive relation? 
b) What is a symmetric relation? 
c) What is an antisymmetric relation? 
d) What is a transitive relation? 
3. Give an example of a relation on the set {1, 2, 3, 4} that is 
a) reflexive, symmetric, and not transitive. 
b) not reflexive, symmetric, and transitive. 
c) reflexive, antisymmetric, and not transitive. 
d) reflexive, symmetric, and transitive. 
e) reflexive, antisymmetric, and transitive. 


4. a) How many reflexive relations are there on a set with n 
elements? 


b) How many symmetric relations are there on a set with 
n elements? 


c) How many antisymmetric relations are there on a set 
with n elements? 


5. a) Explain how an n-ary relation can be used to represent 
information about students at a university. 


b) How can the 5-ary relation containing names of stu- 
dents, their addresses, telephone numbers, majors, and 
grade point averages be used to form a 3-ary relation 
containing the names of students, their majors, and 
their grade point averages? 


compatible total ordering for a partial ordering: a total or- 
dering that contains the given partial ordering 

topological sort: the construction of a total ordering compat- 
ible with a given partial ordering 


RESULTS 


The reflexive closure of arelation R on the set A equals RU A, 
where A = {(a,a) | a € A}. 

The symmetric closure of a relation R on the set A equals 
RU R™!, where R~! = {(b, a) | (a,b) € R}. 

The transitive closure of a relation equals the connectivity re- 
lation formed from this relation. 

Warshall’s algorithm for finding the transitive closure of a re- 
lation 

Let R be an equivalence relation. Then the following three 
statements are equivalent: (1) a R b; (2) [alr N [blr 4 OY; 
(3) [alr = [P]r. 

The equivalence classes of an equivalence relation on a set A 
form a partition of A. Conversely, an equivalence relation 
can be constructed from any partition so that the equivalence 
classes are the subsets in the partition. 

The principle of well-ordered induction 

The topological sorting algorithm 


c) How can the 4-ary relation containing names of stu- 
dents, their addresses, telephone numbers, and majors 
and the 4-ary relation containing names of students, 
their student numbers, majors, and numbers of credit 
hours be combined into a single n-ary relation? 


6. a) Explain how to use a zero—one matrix to represent a 
relation on a finite set. 
b) Explain how to use the zero—one matrix representing a 
relation to determine whether the relation is reflexive, 
symmetric, and/or antisymmetric. 


7. a) Explain how to use a directed graph to represent a re- 
lation on a finite set. 

b) Explain how to use the directed graph representing a 
relation to determine whether a relation is reflexive, 
symmetric, and/or antisymmetric. 

8. a) Define the reflexive closure and the symmetric closure 
of a relation. 

b) How can you construct the reflexive closure of a rela- 
tion? 

c) How can you construct the symmetric closure of a re- 
lation? 

d) Find the reflexive closure and the symmetric closure 
of the relation {(1, 2), (2, 3), (2, 4), (3, 1)} on the set 
{1, 2, 3, 4}. 

9. a) Define the transitive closure of a relation. 

b) Can the transitive closure of a relation be obtained by 
including all pairs (a, c) such that (a, b) and (b, c) be- 
long to the relation? 


10. 


11. 


12. 


13. 


14. 


15. 


c) Describe two algorithms for finding the transitive clo- 
sure of a relation. 

d) Find the transitive closure of the relation 
{(1,1), (1,3), (2,1), (2,3), (2,4), (3,2), 3,4), (4,D}- 

a) Define an equivalence relation. 

b) Which relations on the set {a, b, c, d} are equivalence 
relations and contain (a, b) and (b, d)? 

a) Show that congruence modulo m is an equivalence re- 
lation whenever m is a positive integer. 

b) Show that the relation {(a, b) | a = +b (mod 7)} is an 
equivalence relation on the set of integers. 


a) What are the equivalence classes of an equivalence re- 
lation? 

b) What are the equivalence classes of the “congruent 
modulo 5” relation? 

c) What are the equivalence classes of the equivalence 
relation in Question 11(b)? 

Explain the relationship between equivalence relations on 

a set and partitions of this set. 

a) Define a partial ordering. 

b) Show that the divisibility relation on the set of positive 
integers is a partial order. 

Explain how partial orderings on the sets A; and Az can 

be used to define a partial ordering on the set Ay x Ao. 


16. 


17. 


18. 


19. 


20. 
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a) Explain how to construct the Hasse diagram of a partial 
order on a finite set. 

b) Draw the Hasse diagram of the divisibility relation on 
the set {2, 3, 5,9, 12, 15, 18}. 

a) Define a maximal element of a poset and the greatest 
element of a poset. 

b) Give an example of a poset that has three maximal 
elements. 

c) Give an example of a poset with a greatest element. 


a) Define a lattice. 

b) Give an example of a poset with five elements that is 
a lattice and an example of a poset with five elements 
that is not a lattice. 

a) Show that every finite subset of a lattice has a greatest 
lower bound and a least upper bound. 

b) Show that every lattice with a finite number of elements 
has a least element and a greatest element. 


a) Define a well-ordered set. 

b) Describe an algorithm for producing a totally ordered 
set compatible with a given partially ordered set. 

c) Explain how the algorithm from (b) can be used to or- 
der the tasks in a project if tasks are done one at a time 
and each task can be done only after one or more of 
the other tasks have been completed. 


Supplementary Exercises 


1. 


Let S be the set of all strings of English letters. Determine 
whether these relations are reflexive, irreflexive, symmet- 
ric, antisymmetric, and/or transitive. 

a) R,; = {(a, b) | a and b have no letters in common} 
b) Ro = {(a, b) | a and b are not the same length} 

c) R3 = {(a, b) | a is longer than b} 


. Construct a relation on the set {a, b, c, d} that is 


a) reflexive, symmetric, but not transitive. 

b) irreflexive, symmetric, and transitive. 

c) irreflexive, antisymmetric, and not transitive. 

d) reflexive, neither symmetric nor antisymmetric, and 
transitive. 

e) neither reflexive, irreflexive, symmetric, antisymmet- 
ric, nor transitive. 


. Show that the relation R on ZxZ defined by 


(a, b) R (c, d) if and only if a+d = b+ c is an equiva- 
lence relation. 


. Show that a subset of an antisymmetric relation is also 


antisymmetric. 


. Let R be areflexive relation onaset A. Show that R C R?. 
. Suppose that R; and R>2 are reflexive relations on a set A. 


Show that R; ® R2 is irreflexive. 


. Suppose that R; and R2 are reflexive relations on a set A. 


Is Ry O R>2 also reflexive? Is Rj U Ro also reflexive? 


. Suppose that R is a symmetric relation on a set A. Is R 


also symmetric? 


10 


11 


12 


13 


14 


. Let Ry and Ro be symmetric relations. Is Rj M R2 also 
symmetric? Is Rj U R2 also symmetric? 


. Arelation R is called circular if aRb and bRc imply that 
cRa. Show that R is reflexive and circular if and only if 
it is an equivalence relation. 


. Show that a primary key in an n-ary relation is a primary 
key in any projection of this relation that contains this key 
as one of its fields. 


. Is the primary key in an n-ary relation also a primary 
key in a larger relation obtained by taking the join of this 
relation with a second relation? 


. Show that the reflexive closure of the symmetric closure 
of a relation is the same as the symmetric closure of its 
reflexive closure. 


. Let R be the relation on the set of all mathematicians that 
contains the ordered pair (a, b) if and only if a and b have 
written a published mathematical paper together. 


a) Describe the relation R?. 

b) Describe the relation R*. 

c) The Erd6és number of a mathematician is 1 if this 
=~ Mathematician wrote a paper with the prolific Hun- 
@ garian mathematician Paul Erdés, it is 2 if this math- 

ematician did not write a joint paper with Erdés but 
wrote a joint paper with someone who wrote a joint 
paper with Erdés, and so on (except that the Erdés 
number of Erdés himself is 0). Give a definition of the 
Erdés number in terms of paths in R. 
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15. 


16. 


17. 


18. 


*19, 


20. 


a) Give an example to show that the transitive closure a) {(x, y) | x and y have the same sign of the zodiac} 
of the symmetric closure of a relation is not necessar- b) {(x, y) | x and y were born in the same year} 
ily the same as the symmetric closure of the transitive ; ; 
closure of this relation. c) {(x, y) | x and y have been in the same city} 

b) Show, however, that the transitive closure of the sym- *21. How many different equivalence relations with exactly 
metric closure of arelation must contain the symmetric three different equivalence classes are there on a set with 
closure of the transitive closure of this relation. five elements? 

a) Let S be the set of subroutines of a computer pro- 22. Show that {(x, y) | x — y € Q} is an equivalence relation 
gram. Define the relation R by PR Q if subroutine P on the set of real numbers, where Q denotes the set of 
calls subroutine Q during its execution. Describe the rational numbers. What are [1], [ it ], and [7]? 
transitive closure of R. 

b) For which subroutines P does (P, P) belong to the 23. Suppose that P; ={Aj,A2,...,Am} and P2= 
transitive closure of R? {B1, Bo,..., Bn} are both partitions of the set S. Show 

c) Describe the reflexive closure of the transitive closure that the collection of nonempty subsets of the form 
of R. A; /O B; is a partition of S that is a refinement of both P; 

Suppose that R and S are relations on a set A with R C S and P> (see the preamble to Exercise 49 of Section 9.5). 

such that the closures of R and S with respect to a prop- * 24. Show that the transitive closure of the symmetric closure 

erty P both exist. Show that the closure of R with respect of the reflexive closure of a relation R is the smallest 

to P is a subset of the closure of S with respect to P. equivalence relation that contains R. 


Show that the symmetric closure of the union of two re- 
lations is the union of their symmetric closures. 

Devise an algorithm, based on the concept of interior ver- 
tices, that finds the length of the longest path between two 


25. Let R(S) be the set of all relations on a set S. Define the 
relation =< on R(S) by Ri =< R2 if Ri C Ro, where R 
and R» are relations on S. Show that (R(S), = ) is a poset. 


vertices in a directed graph, or determines that there are 26. Let P(S) be the set of all partitions of the set S. Define the 
arbitrarily long paths between these vertices. relation = on P(S) by P, = P> if P, is a refinement of 
Which of these are equivalence relations on the set of all Po (see Exercise 49 of Section 9.5). Show that (P(S), = ) 
people? is a poset. 


PAUL ERDOS (1913-1996) Paul Erdés, born in Budapest, Hungary, was the son of two high school mathe- 
matics teachers. He was a child prodigy; at age 3 he could multiply three-digit numbers in his head, and at 4 he 
discovered negative numbers on his own. Because his mother did not want to expose him to contagious diseases, 
he was mostly home-schooled. At 17 Erdés entered Edtvés University, graduating four years later with a Ph.D. 
in mathematics. After graduating he spent four years at Manchester, England, on a postdoctoral fellowship. In 
1938 he went to the United States because of the difficult political situation in Hungary, especially for Jews. 
He spent much of his time in the United States, except for 1954 to 1962, when he was banned as part of the 
paranoia of the McCarthy era. He also spent considerable time in Israel. 

Erdés made many significant contributions to combinatorics and to number theory. One of the discoveries 
of which he was most proud is his elementary proof (in the sense that it does not use any complex analysis) of the prime number 
theorem, which provides an estimate for the number of primes not exceeding a fixed positive integer. He also participated in the 
modern development of the Ramsey theory. 

Erdés traveled extensively throughout the world to work with other mathematicians, visiting conferences, universities, and 
research laboratories. He had no permanent home. He devoted himself almost entirely to mathematics, traveling from one mathe- 
matician to the next, proclaiming “My brain is open.” Erdés was the author or coauthor of more than 1500 papers and had more 
than 500 coauthors. Copies of his articles are kept by Ron Graham, a famous discrete mathematician with whom he collaborated 
extensively and who took care of many of his worldly needs. 

Erdés offered rewards, ranging from $10 to $10,000, for the solution of problems that he found particularly interesting, with the 
size of the reward depending on the difficulty of the problem. He paid out close to $4000. Erdés had his own special language, using 
such terms as “epsilon” (child), “boss” (woman), “slave” (man), “captured” (married), “liberated” (divorced), “Supreme Fascist” 
(God), “Sam” (United States), and “Joe” (Soviet Union). Although he was curious about many things, he concentrated almost all 
his energy on mathematical research. He had no hobbies and no full-time job. He never married and apparently remained celibate. 
Erdés was extremely generous, donating much of the money he collected from prizes, awards, and stipends for scholarships and to 
worthwhile causes. He traveled extremely lightly and did not like having many material possessions. 


27. Schedule the tasks needed to cook a Chinese meal by 
specifying their order, if the Hasse diagram representing 


these tasks is as shown here. 
Serve 


Arrange on platters 


Make 
garnishes 


vegetables 


Cut ginger 
and garlic 


Buy groceries 


Find recipe 


A subset of a poset such that every two elements of this sub- 
set are comparable is called a chain. A subset of a poset is 
called an antichain if every two elements of this subset are 
incomparable. 


28. Find all chains in the posets with the Hasse diagrams 
shown in Exercises 25—27 in Section 9.6. 


29. Find all antichains in the posets with the Hasse diagrams 
shown in Exercises 25—27 in Section 9.6. 


30. Find an antichain with the greatest number of elements 
in the poset with the Hasse diagram of Exercise 32 in 
Section 9.6. 


31. Show that every maximal chain in a finite poset (S, = ) 
contains a minimal element of S. (A maximal chain is a 
chain that is not a subset of a larger chain.) 


** 32. Show that every finite poset can be partitioned into k 


chains, where k is the largest number of elements in an 
antichain in this poset. 


* 33. Show that in any group of mn + | people there is either a 
list of m + 1 people where a person in the list (except for 
the first person listed) is a descendant of the previous per- 
son on the list, or there are n + 1 people such that none of 
these people is a descendant of any of the other people. 
[Hint: Use Exercise 32.] 


Suppose that (S, = ) is a well-founded partially ordered set. 
The principle of well-founded induction states that P(x) is 
true for all x € Sif Vx(Vy(y ~ x > P(y)) > P(x)). 


34. Show that no separate basis case is needed for the prin- 
ciple of well-founded induction. That is, P (uw) is true for 
all minimal elements u in Sif Vx(Vy(y ~ x > P(y)) > 
P(x)). 
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* 35. Show that the principle of well-founded induction is valid. 


A relation R on a set A is a quasi-ordering on A if R is 
reflexive and transitive. 


36. Let R be the relation on the set of all functions from 
Z* to Z* such that (f, g) belongs to R if and only if f 
is O(g). Show that R is a quasi-ordering. 


37. Let R be a quasi-ordering on a set A. Show that RN R7! 
is an equivalence relation. 


* 38. Let R be a quasi-ordering and let S be the relation on the 
set of equivalence classes of RM R—! such that (C, D) 
belongs to S, where C and D are equivalence classes 
of R, if and only if there are elements c of C and d of 
D such that (c, d) belongs to R. Show that S is a partial 
ordering. 

Let L be a lattice. Define the meet (A) and join (V) operations 
by x A y = glb(x, y) and x Vv y = lub(x, y). 
39. Show that the following properties hold for all elements 
x, y, and z of a lattice L. 
a) xAy=yAx and x Vy=yVvx (commutative 
laws) 
b) @Ay)AZ=xXA(VAzZ) and WV y)VzZ= xv 
(y V z) (associative laws) 
c) x A(x Vy) =x and x V(xA y) =x (absorption 
laws) 
d) x Ax =x and x V x = x (idempotent laws) 
40. Show that if x and y are elements of a lattice L, then 
xVy=vyifandonlyifxA y=x. 
A lattice L is bounded if it has both an upper bound, de- 
noted by 1, such that x = 1 for all x € L anda lower bound, 
denoted by 0, such that 0 = x for all x € L. 


41. Show that if Z is a bounded lattice with upper bound 
1 and lower bound 0 then these properties hold for all 
elements x € L. 
a)xvl=1 b) xAl=x 
c) xVO=x d) xA0=0 

42. Show that every finite lattice is bounded. 

A lattice is called distributive if x V (yA z)=(xV y)A 

(x Vz) andx A (y Vz) = (*A y) V XA Zz) forall x, y, and 

zinL. 

* 43, Give an example of a lattice that is not distributive. 


44. Show that the lattice (P(S), ©) where P(S) is the power 
set of a finite set § is distributive. 


45. Is the lattice (Z*, |) distributive? 

The complement of an element a of a bounded lattice L with 
upper bound | and lower bound 0 is an element b such that 
aVb=1andaAb=0. Such a lattice is complemented if 
every element of the lattice has a complement. 


46. Give an example of a finite lattice where at least one el- 
ement has more than one complement and at least one 
element has no complement. 


47. Show that the lattice (P(S), ©) where P(S) is the power 
set of a finite set S is complemented. 
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* 48. Show that if L is a finite distributive lattice, then an ele- 
ment of L has at most one complement. 

The game of Chomp, introduced in Example 12 in Section 1.8, 
can be generalized for play on any finite partially ordered set 
(S, <) with a least element a. In this game, a move consists of 
selecting an element x in S and removing x and all elements 
larger than it from S. The loser is the player who is forced to 
select the least element a. 


Computer Projects 


Write programs with these input and output. 


= 


Given the matrix representing a relation on a finite set, de- 
termine whether the relation is reflexive and/or irreflexive. 
Given the matrix representing a relation on a finite set, 
determine whether the relation is symmetric and/or anti- 
symmetric. 

3. Given the matrix representing a relation on a finite set, 
determine whether the relation is transitive. 


N 


4. Given a positive integer n, display all the relations on a set 
with n elements. 
*5, Given a positive integer n, determine the number of tran- 
sitive relations on a set with n elements. 
*6. Given a positive integer n, determine the number of equiv- 
alence relations on a set with n elements. 
*7, Given a positive integer n, display all the equivalence re- 
lations on the set of the n smallest positive integers. 
Given an n-ary relation, find the projection of this relation 
when specified fields are deleted. 
9. Given an m-ary relation and an n-ary relation, and a set of 
common fields, find the join of these relations with respect 
to these common fields. 


ad 


Computations and Explorations 


49. Show that the game of Chomp with cookies arranged in 
an m x n rectangular grid, described in Example 12 in 
Section 1.8, is the same as the game of Chomp on the 
poset (S, |), where S is the set of all positive integers that 
divide p”—!q"—!, where p and g are distinct primes. 

50. Show that if (S, <) has a greatest element b, then a win- 
ning strategy for Chomp on this poset exists. [Hint: Gen- 
eralize the argument in Example 12 in Section 1.8.] 


10. Given the matrix representing a relation on a finite set, 
find the matrix representing the reflexive closure of this 
relation. 


11. Given the matrix representing a relation on a finite set, 
find the matrix representing the symmetric closure of this 
relation. 


12. Given the matrix representing a relation on a finite set, 
find the matrix representing the transitive closure of this 
relation by computing the join of the Boolean powers of 
the matrix representing the relation. 


13. Given the matrix representing a relation on a finite set, 
find the matrix representing the transitive closure of this 
relation using Warshall’s algorithm. 


14. Given the matrix representing a relation on a finite set, find 
the matrix representing the smallest equivalence relation 
containing this relation. 


15. Given a partial ordering on a finite set, find a total ordering 
compatible with it using topological sorting. 


Use a computational program or programs you have written to do these exercises. 


1. Display all the different relations on a set with four ele- 
ments. 

2. Display all the different reflexive and symmetric relations 
on a set with six elements. 

3. Display all the reflexive and transitive relations on a set 
with five elements. 

*4, Determine how many transitive relations there are on a set 

with n elements for all positive integers n with n < 7. 

5. Find the transitive closure of a relation of your choice on 
a set with at least 20 elements. Either use a relation that 


corresponds to direct links in a particular transportation 
or communications network or use a randomly generated 
relation. 
6. Compute the number of different equivalence relations ona 
set with n elements for all positive integers n not exceeding 
20. 
7. Display all the equivalence relations on a set with seven 
elements. 
* 8. Display all the partial orders on a set with five elements. 
*9,. Display all the lattices on a set with five elements. 


Writing Projects 


Respond to these with essays using outside sources. 
1. 


2. 


Discuss the concept of a fuzzy relation. How are fuzzy 
relations used? 


Describe the basic principles of relational databases, go- 
ing beyond what was covered in Section 9.2. How widely 
used are relational databases as compared with other types 
of databases? 


. Look up the original papers by Warshall and by Roy (in 


French) in which they develop algorithms for finding tran- 
sitive closures. Discuss their approaches. Why do you 
suppose that what we call Warshall’s algorithm was dis- 
covered independently by more than one person? 


. Describe how equivalence classes can be used to define 


the rational numbers as classes of pairs of integers and 
how the basic arithmetic operations on rational numbers 
can be defined following this approach. (See Exercise 40 
in Section 9.5.) 


10. 
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. Explain how Helmut Hasse used what we now call Hasse 


diagrams. 


. Describe some of the mechanisms used to enforce infor- 


mation flow policies in computer operating systems. 


. Discuss the use of the Program Evaluation and Review 


Technique (PERT) to schedule the tasks of a large com- 
plicated project. How widely is PERT used? 


. Discuss the use of the Critical Path Method (CPM) to find 


the shortest time for the completion of a project. How 
widely is CPM used? 


. Discuss the concept of duality in a lattice. Explain how 


duality can be used to establish new results. 

Explain what is meant by a modular lattice. Describe 
some of the properties of modular lattices and describe 
how modular lattices arise in the study of projective ge- 
ometry. 


10.1 Graphs and 
Graph Models 


10.2 Graph 
Terminology 
and Special 
Types of 
Graphs 


10.3 Representing 
Graphs and 
Graph 
Isomorphism 


10.4 Connectivity 


10.5 Euler and 
Hamilton 
Paths 


10.6 Shortest-Path 
Problems 


10.7 Planar Graphs 


10.8 Graph 
Coloring 


Graphs 


( . raphs are discrete structures consisting of vertices and edges that connect these vertices. 

There are different kinds of graphs, depending on whether edges have directions, whether 
multiple edges can connect the same pair of vertices, and whether loops are allowed. Problems 
in almost every conceivable discipline can be solved using graph models. We will give examples 
to illustrate how graphs are used as models in a variety of areas. For instance, we will show how 
graphs are used to represent the competition of different species in an ecological niche, how 
graphs are used to represent who influences whom in an organization, and how graphs are used 
to represent the outcomes of round-robin tournaments. We will describe how graphs can be used 
to model acquaintanceships between people, collaboration between researchers, telephone calls 
between telephone numbers, and links between websites. We will show how graphs can be used 
to model roadmaps and the assignment of jobs to employees of an organization. 

Using graph models, we can determine whether it is possible to walk down all the streets 
in a city without going down a street twice, and we can find the number of colors needed to 
color the regions of a map. Graphs can be used to determine whether a circuit can be imple- 
mented on a planar circuit board. We can distinguish between two chemical compounds with the 
same molecular formula but different structures using graphs. We can determine whether two 
computers are connected by a communications link using graph models of computer networks. 
Graphs with weights assigned to their edges can be used to solve problems such as finding the 
shortest path between two cities in a transportation network. We can also use graphs to schedule 
exams and assign channels to television stations. This chapter will introduce the basic concepts 
of graph theory and present many different graph models. To solve the wide variety of problems 
that can be studied using graphs, we will introduce many different graph algorithms. We will 
also study the complexity of these algorithms. 


Graphs and Graph Models 


DEFINITION 1 


We begin with the definition of a graph. 


A graph G = (V, E) consists of V, a nonempty set of vertices (or nodes) and E, a set of 
edges. Each edge has either one or two vertices associated with it, called its endpoints. An 
edge is said to connect its endpoints. 


Remark: The set of vertices V of a graph G may be infinite. A graph with an infinite vertex 
set or an infinite number of edges is called an infinite graph, and in comparison, a graph with 
a finite vertex set and a finite edge set is called a finite graph. In this book we will usually 
consider only finite graphs. 


Now suppose that a network is made up of data centers and communication links between 
computers. We can represent the location of each data center by a point and each communications 
link by a line segment, as shown in Figure 1. 

This computer network can be modeled using a graph in which the vertices of the graph 
represent the data centers and the edges represent communication links. In general, we visualize 


641 


642 


10 / Graphs 


Detroit 


New York 
San Francisco 


Washington 
Denver s 


Los Angeles 


FIGURE 1 A Computer Network. 


graphs by using points to represent vertices and line segments, possibly curved, to represent 
edges, where the endpoints of a line segment representing an edge are the points representing 
the endpoints of the edge. When we draw a graph, we generally try to draw edges so that they do 
not cross. However, this is not necessary because any depiction using points to represent vertices 
and any form of connection between vertices can be used. Indeed, there are some graphs that 
cannot be drawn in the plane without edges crossing (see Section 10.7). The key point is that 
the way we draw a graph is arbitrary, as long as the correct connections between vertices are 
depicted. 

Note that each edge of the graph representing this computer network connects two different 
vertices. That is, no edge connects a vertex to itself. Furthermore, no two different edges connect 
the same pair of vertices. A graph in which each edge connects two different vertices and where 
no two edges connect the same pair of vertices is called a simple graph. Note that in a simple 
graph, each edge is associated to an unordered pair of vertices, and no other edge is associated 
to this same edge. Consequently, when there is an edge of a simple graph associated to {u, v}, 
we can also say, without possible confusion, that {u, v} is an edge of the graph. 

A computer network may contain multiple links between data centers, as shown in Figure 2. 
To model such networks we need graphs that have more than one edge connecting the same 
pair of vertices. Graphs that may have multiple edges connecting the same vertices are called 
multigraphs. When there are m different edges associated to the same unordered pair of vertices 
{u, v}, we also say that {u, v} is an edge of multiplicity m. That is, we can think of this set of 
edges as m different copies of an edge {u, v}. 
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FIGURE 2 A Computer Network with Multiple Links between Data Centers. 


Sometimes a communications link connects a data center with itself, perhaps a feedback 
loop for diagnostic purposes. Such a network is illustrated in Figure 3. To model this network we 
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FIGURE 3 A Computer Network with Diagnostic Links. 
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FIGURE 4 A Communications Network with One-Way Communications Links. 


need to include edges that connect a vertex to itself. Such edges are called loops, and sometimes 
we may even have more than one loop at a vertex. Graphs that may include loops, and possibly 
multiple edges connecting the same pair of vertices or a vertex to itself, are sometimes called 
pseudographs. 

So far the graphs we have introduced are undirected graphs. Their edges are also said 
to be undirected. However, to construct a graph model, we may find it necessary to assign 
directions to the edges of a graph. For example, in a computer network, some links may operate 
in only one direction (such links are called single duplex lines). This may be the case if there is 
a large amount of traffic sent to some data centers, with little or no traffic going in the opposite 
direction. Such a network is shown in Figure 4. 

To model such a computer network we use a directed graph. Each edge of a directed graph 
is associated to an ordered pair. The definition of directed graph we give here is more general 
than the one we used in Chapter 9, where we used directed graphs to represent relations. 


A directed graph (or digraph) (V, E) consists of a nonempty set of vertices V and a set of 
directed edges (or arcs) E.. Each directed edge is associated with an ordered pair of vertices. 
The directed edge associated with the ordered pair (u, v) is said to start at u and end at v. 


When we depict a directed graph with a line drawing, we use an arrow pointing from u to v to 
indicate the direction of an edge that starts at u and ends at v. A directed graph may contain loops 
and it may contain multiple directed edges that start and end at the same vertices. A directed graph 
may also contain directed edges that connect vertices u and v in both directions; that is, when a 
digraph contains an edge from u to v, it may also contain one or more edges from v to uv. Note that 
we obtain a directed graph when we assign a direction to each edge in an undirected graph. When 
a directed graph has no loops and has no multiple directed edges, it is called a simple directed 
graph. Because a simple directed graph has at most one edge associated to each ordered pair 
of vertices (u, v), we call (u, v) an edge if there is an edge associated to it in the graph. 

In some computer networks, multiple communication links between two data centers may 
be present, as illustrated in Figure 5. Directed graphs that may have multiple directed edges 
from a vertex to a second (possibly the same) vertex are used to model such networks. We called 
such graphs directed multigraphs. When there are m directed edges, each associated to an 
ordered pair of vertices (u, v), we say that (u, v) is an edge of multiplicity m. 
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FIGURE 5 A Computer Network with Multiple One-Way Links. 


644 10/Graphs 


TABLE 1 Graph Terminology. 


Simple graph Undirected 

Multigraph Undirected 
Pseudograph Undirected 

Simple directed graph Directed 

Directed multigraph Directed 

Mixed graph Directed and undirected 


Links 


Links 


Can you find a subject to 
which graph theory has 
not been applied? 


For some models we may need a graph where some edges are undirected, while others are 
directed. A graph with both directed and undirected edges is called a mixed graph. For example, 
a mixed graph might be used to model a computer network containing links that operate in both 
directions and other links that operate only in one direction. 

This terminology for the various types of graphs is summarized in Table 1. We will some- 
times use the term graph as a general term to describe graphs with directed or undirected edges 
(or both), with or without loops, and with or without multiple edges. At other times, when the 
context is clear, we will use the term graph to refer only to undirected graphs. 

Because of the relatively modern interest in graph theory, and because it has applications toa 
wide variety of disciplines, many different terminologies of graph theory have been introduced. 
The reader should determine how such terms are being used whenever they are encountered. 
The terminology used by mathematicians to describe graphs has been increasingly standardized, 
but the terminology used to discuss graphs when they are used in other disciplines is still quite 
varied. Although the terminology used to describe graphs may vary, three key questions can 
help us understand the structure of a graph: 


@ Are the edges of the graph undirected or directed (or both)? 


@ Ifthe graph is undirected, are multiple edges present that connect the same pair of vertices? 
If the graph is directed, are multiple directed edges present? 


® Are loops present? 


Answering such questions helps us understand graphs. It is less important to remember the 
particular terminology used. 


Graph Models 


Graphs are used in a wide variety of models. We began this section by describing how to construct 
graph models of communications networks linking data centers. We will complete this section 
by describing some diverse graph models for some interesting applications. We will return to 
many of these applications later in this chapter and in Chapter 11. We will introduce additional 
graph models in subsequent sections of this and later chapters. Also, recall that directed graph 
models for some applications were introduced in Chapter 9. When we build a graph model, we 
need to make sure that we have correctly answered the three key questions we posed about the 
structure of a graph. 


SOCIAL NETWORKS Graphs are extensively used to model social structures based on dif- 
ferent kinds of relationships between people or groups of people. These social structures, and the 
graphs that represent them, are known as social networks. In these graph models, individuals 
or organizations are represented by vertices; relationships between individuals or organizations 
are represented by edges. The study of social networks is an extremely active multidisciplinary 
area, and many different types of relationships between people have been studied using them. 
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FIGURE 6 An Acquaintanceship Graph. FIGURE7 An Influence Graph. 


EXAMPLE 1 


Links 


EXAMPLE 2 


EXAMPLE 3 


Links 


We will introduce some of the most commonly studied social networks here. More information 
about social networks can be found in [Ne10] and [EaK110]. 


Acquaintanceship and Friendship Graphs We can use a simple graph to represent whether 
two people know each other, that is, whether they are acquainted, or whether they are friends 
(either in the real world in the virtual world via a social networking site such as Facebook). 
Each person in a particular group of people is represented by a vertex. An undirected edge is 
used to connect two people when these people know each other, when we are concerned only 
with acquaintanceship, or whether they are friends. No multiple edges and usually no loops are 
used. (If we want to include the notion of self-knowledge, we would include loops.) A small 
acquaintanceship graph is shown in Figure 6. The acquaintanceship graph of all people in the 
world has more than six billion vertices and probably more than one trillion edges! We will 
discuss this graph further in Section 10.4. < 


Influence Graphs In studies of group behavior it is observed that certain people can influence 
the thinking of others. A directed graph called an influence graph can be used to model this 
behavior. Each person of the group is represented by a vertex. There is a directed edge from 
vertex a to vertex b when the person represented by vertex a can influence the person represented 
by vertex b. This graph does not contain loops and it does not contain multiple directed edges. 
An example of an influence graph for members of a group is shown in Figure 7. In the group 
modeled by this influence graph, Deborah cannot be influenced, but she can influence Brian, 
Fred, and Linda. Also, Yvonne and Brian can influence each other. < 


Collaboration Graphs A collaboration graph is used to model social networks where two 
people are related by working together in a particular way. Collaboration graphs are simple 
graphs, as edges in these graphs are undirected and there are no multiple edges or loops. Vertices 
in these graphs represent people; two people are connected by an undirected edge when the 
people have collaborated. There are no loops nor multiple edges in these graphs. The Hollywood 
graph is a collaborator graph that represents actors by vertices and connects two actors with 
an edge if they have worked together on a movie or television show. The Hollywood graph is a 
huge graph with more than 1.5 million vertices (as of early 2011). We will discuss some aspects 
of the Hollywood graph later in Section 10.4. 

In an academic collaboration graph, vertices represent people (perhaps restricted to mem- 
bers of a certain academic community), and edges link two people if they have jointly published 
a paper. The collaboration graph for people who have published research papers in mathematics 
was found in 2004 to have more than 400,000 vertices and 675,000 edges, and these numbers 
have grown considerably since then. We will have more to say about this graph in Section 10.4. 
Collaboration graphs have also been used in sports, where two professional athletes are consid- 
ered to have collaborated if they have ever played on the same team during a regular season of 
their sport. < 
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EXAMPLE 4 
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COMMUNICATION NETWORKS Wecan model different communications networks using 
vertices to represent devices and edges to represent the particular type of communications links 
of interest. We have already modeled a data network in the first part of this section. 


Call Graphs Graphs can be used to model telephone calls made in a network, such as a long- 
distance telephone network. In particular, a directed multigraph can be used to model calls where 
each telephone number is represented by a vertex and each telephone call is represented by a 
directed edge. The edge representing a call starts at the telephone number from which the call 
was made and ends at the telephone number to which the call was made. We need directed edges 
because the direction in which the call is made matters. We need multiple directed edges because 
we want to represent each call made from a particular telephone number to a second number. 

A small telephone call graph is displayed in Figure 8(a), representing seven telephone 
numbers. This graph shows, for instance, that three calls have been made from 732-555-1234 
to 732-555-9876 and two in the other direction, but no calls have been made from 732-555-4444 
to any of the other six numbers except 732-555-0011. When we care only whether there has been 
a call connecting two telephone numbers, we use an undirected graph with an edge connecting 
telephone numbers when there has been a call between these numbers. This version of the call 
graph is displayed in Figure 8(b). 

Call graphs that model actual calling activities can be huge. For example, one call graph 
studied at AT&T, which models calls during 20 days, has about 290 million vertices and 4 billion 
edges. We will discuss call graphs further in Section 10.4. < 


INFORMATION NETWORKS Graphs can be used to model various networks that link 
particular types of information. Here, we will describe how to model the World Wide Web using 
a graph. We will also describe how to use a graph to model the citations in different types of 
documents. 


The Web Graph The World Wide Web can be modeled as a directed graph where each Web 
page is represented by a vertex and where an edge starts at the Web page a and ends at the 
Web page D if there is a link on a pointing to b. Because new Web pages are created and others 
removed somewhere on the Web almost every second, the Web graph changes on an almost 
continual basis. Many people are studying the properties of the Web graph to better understand 
the nature of the Web. We will return to Web graphs in Section 10.4, and in Chapter 11 we will 
explain how the Web graph is used by the Web crawlers that search engines use to create indexes 
of Web pages. 


Citation Graphs Graphs can be used to represent citations in different types of documents, 
including academic papers, patents, and legal opinions. In such graphs, each document is rep- 
resented by a vertex, and there is an edge from one document to a second document if the 
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FIGURE 8 A Call Graph. 
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first document cites the second in its citation list. (In an academic paper, the citation list is the 
bibliography, or list of references; in a patent it is the list of previous patents that are cited; and 
in a legal opinion it is the list of previous opinions cited.) A citation graph is a directed graph 
without loops or multiple edges. 


SOFTWARE DESIGN APPLICATIONS Graph models are useful tools in the design of 
software. We will briefly describe two of these models here. 


Module Dependency Graphs One of the most important tasks in designing software is how to 
structure a program into different parts, or modules. Understanding how the different modules of 
a program interact is essential not only for program design, but also for testing and maintenance 
of the resulting software. A module dependency graph provides a useful tool for understanding 
how different modules of a program interact. In a program dependency graph, each module is 
represented by a vertex. There is a directed edge from a module to a second module if the second 
module depends on the first. An example of a program dependency graph for a web browser is 
shown in Figure 9. 


Precedence Graphs and Concurrent Processing Computer programs can be executed more 
rapidly by executing certain statements concurrently. It is important not to execute a statement 
that requires results of statements not yet executed. The dependence of statements on previous 
statements can be represented by a directed graph. Each statement is represented by a vertex, 
and there is an edge from one statement to a second statement if the second statement cannot 
be executed before the first statement. This resulting graph is called a precedence graph. A 
computer program and its graph are displayed in Figure 10. For instance, the graph shows that 
statement S5 cannot be executed before statements $1, $2, and S4 are executed. < 


TRANSPORTATION NETWORKS We can use graphs to model many different types of 
transportation networks, including road, air, and rail networks, as well shipping networks. 


Airline Routes We can model airline networks by representing each airport by a vertex. In 
particular, we can model all the flights by a particular airline each day using a directed edge 
to represent each flight, going from the vertex representing the departure airport to the vertex 
representing the destination airport. The resulting graph will generally be a directed multigraph, 
as there may be multiple flights from one airport to some other airport during the same day. 4 


Road Networks Graphs can be used to model road networks. In such models, vertices repre- 
sent intersections and edges represent roads. When all roads are two-way and there is at most 
one road connecting two intersections, we can use a simple undirected graph to model the road 
network. However, we will often want to model road networks when some roads are one-way 
and when there may be more than one road between two intersections. To build such models, 
we use undirected edges to represent two-way roads and we use directed edges to represent 
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FIGURE 9 A Module Dependency Graph. FIGURE 10 A Precedence Graph. 
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one-way roads. Multiple undirected edges represent multiple two-way roads connecting the 
same two intersections. Multiple directed edges represent multiple one-way roads that start at 
one intersection and end at a second intersection. Loops represent loop roads. Mixed graphs are 
needed to model road networks that include both one-way and two-way roads. 


BIOLOGICAL NETWORKS Many aspects of the biological sciences can be modeled using 
graphs. 


Niche Overlap Graphs in Ecology Graphs are used in many models involving the interaction 
of different species of animals. For instance, the competition between species in an ecosystem 
can be modeled using a niche overlap graph. Each species is represented by a vertex. An 
undirected edge connects two vertices if the two species represented by these vertices compete 
(that is, some of the food resources they use are the same). A niche overlap graph is a simple 
graph because no loops or multiple edges are needed in this model. The graph in Figure 11 
models the ecosystem of a forest. We see from this graph that squirrels and raccoons compete 
but that crows and shrews do not. 


Protein Interaction Graphs A protein interaction in a living cell occurs when two or more 
proteins in that cell bind to perform a biological function. Because protein interactions are 
crucial for most biological functions, many scientists work on discovering new proteins and 
understanding interactions between proteins. Protein interactions within a cell can be modeled 
using a protein interaction graph (also called a protein-protein interaction network), an 
undirected graph in which each protein is represented by a vertex, with an edge connecting the 
vertices representing each pair of proteins that interact. It is a challenging problem to determine 
genuine protein interactions in a cell, as experiments often produce false positives, which con- 
clude that two proteins interact when they really do not. Protein interaction graphs can be used 
to deduce important biological information, such as by identifying the most important proteins 
for various functions and the functionality of newly discovered proteins. 

Because there are thousands of different proteins in a typical cell, the protein interaction 
graph of a cell is extremely large and complex. For example, yeast cells have more than 6,000 
proteins, and more than 80,000 interactions between them are known, and human cells have 
more than 100,000 proteins, with perhaps as many as 1,000,000 interactions between them. 
Additional vertices and edges are added to a protein interaction graph when new proteins and 
interactions between proteins are discovered. Because of the complexity of protein interac- 
tion graphs, they are often split into smaller graphs called modules that represent groups of 
proteins that are involved in a particular function of a cell. Figure 12 illustrates a module of 
the protein interaction graph described in [Bo04], comprising the complex of proteins that de- 
grade RNA in human cells. To learn more about protein interaction graphs, see [Bo04], [Ne10], 
and [Hu07]. < 
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FIGURE 11 A Niche Overlap Graph. FIGURE 12 A Module of a Protein Interaction Graph. 
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TOURNAMENTS We now give some examples that show how graphs can also be used to 
model different kinds of tournaments. 

EXAMPLE 13 Round-Robin Tournaments A tournament where each team plays every other team exactly 
once and no ties are allowed is called a round-robin tournament. Such tournaments can be 
modeled using directed graphs where each team is represented by a vertex. Note that (a, b) is 
an edge if team a beats team b. This graph is a simple directed graph, containing no loops or 
multiple directed edges (because no two teams play each other more than once). Such a directed 
graph model is presented in Figure 13. We see that Team | is undefeated in this tournament, 
and Team 3 is winless. < 

EXAMPLE 14  Single-Elimination Tournaments A tournament where each contestant is eliminated after one 
loss is called a single-elimination tournament. Single-elimination tournaments are often used 
in sports, including tennis championships and the yearly NCAA basketball championship. We 
can model such a tournament using a vertex to represent each game and a directed edge to connect 
a game to the next game the winner of this game played in. The graph in Figure 14 represents 
the games played by the final 16 teams in the 2010 NCAA women’s basketball tournament. 4 

Exercises 
1. Draw graph models, stating the type of graph (from Ta- plus a loop for a special sightseeing trip that takes off 
ble 1) used, to represent airline routes where every day and lands in Miami. 

there are four flights from Boston to Newark, two flights d) anedge froma vertex representing acity where a flight 

from Newark to Boston, three flights from Newark to Mi- starts to the vertex representing the city where it ends. 

ami, two flights from Miami to Newark, one flight from e) an edge for each flight from a vertex representing a 

Newark to Detroit, two flights from Detroit to Newark, city where the flight begins to the vertex representing 

three flights from Newark to Washington, two flights from the city where the flight ends. 

Washington to Newark, and one flight from Washington 2. What kind of graph (from Table 1) can be used to model 


to Miami, with 


a) an edge between vertices representing cities that have 
a flight between them (in either direction). 


a highway system between major cities where 


a) there is an edge between the vertices representing 
cities if there is an interstate highway between them? 
b) there is an edge between the vertices representing 


b) an edge between vertices representing cities for each cities for each interstate highway between them? 
flight that operates between them (in either direction). c) there is an edge between the vertices representing 


cities for each interstate highway between them, and 


c) an edge between vertices representing cities for each there is a loop at the vertex representing a city if there 
flight that operates between them (in either direction), is an interstate highway that circles this city? 
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For Exercises 3—9, determine whether the graph shown has 
directed or undirected edges, whether it has multiple edges, 
and whether it has one or more loops. Use your answers to 
determine the type of graph in Table | this graph is. 


3. 


10. 


11. 


12. 


13. 


a b 


4.4 = b 
c d 
6. a b 
} | 
e 
c d 
&.4 : 
d 
b c 


For each undirected graph in Exercises 3-9 that is not 
simple, find a set of edges to remove to make it simple. 


Let G be a simple graph. Show that the relation R on the 
set of vertices of G such that uRv if and only if there 
is an edge associated to {u, v} is a symmetric, irreflexive 
relation on G. 


Let G be an undirected graph with a loop at every vertex. 
Show that the relation R on the set of vertices of G such 
that u Rv if and only if there is an edge associated to {u, v} 
is a symmetric, reflexive relation on G. 


The intersection graph of a collection of sets Aj, 
A2,..., A, is the graph that has a vertex for each of these 
sets and has an edge connecting the vertices representing 
two sets if these sets have a nonempty intersection. Con- 
struct the intersection graph of these collections of sets. 
a) A; = {0, 2, 4, 6, 8}, Az = {0, 1, 2, 3, 4}, 

A3 = {1,3,5, 7,9}, Aq = {5, 6, 7, 8, 9}, 

As = {0, 1, 8, 9} 
b) Aj ={..., —4, —3, —2, —1, 0}, 

Az ={...,-2, —1,0,1,2,...}, 

A3 ={..., —6, —4, —2, 0, 2, 4, 6, ...}, 

Aq = {...,—-5, —3, —1, 1,3, 5, ...}, 

As = {..., —6, —3, 0,3, 6,...} 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


c) Ay = {x |x < 0}, 

A2 = {x | -1 < x < O}, 

A3 = {x |0<-x < ]}, 

Ag = {x | -I <x < ]}, 

As = {x |x > —l]}, 

Aob=R 
Use the niche overlap graph in Figure 11 to determine the 
species that compete with hawks. 
Construct a niche overlap graph for six species of birds, 
where the hermit thrush competes with the robin and 
with the blue jay, the robin also competes with the 
mockingbird, the mockingbird also competes with the 
blue jay, and the nuthatch competes with the hairy wood- 
pecker. 
Draw the acquaintanceship graph that represents that Tom 
and Patricia, Tom and Hope, Tom and Sandy, Tom and 
Amy, Tom and Marika, Jeff and Patricia, Jeff and Mary, 
Patricia and Hope, Amy and Hope, and Amy and Marika 
know each other, but none of the other pairs of people 
listed know each other. 
We can use a graph to represent whether two people were 
alive at the same time. Draw such a graph to represent 
whether each pair of the mathematicians and computer 
scientists with biographies in the first five chapters of 
this book who died before 1900 were contemporaneous. 
(Assume two people lived at the same time if they were 
alive during the same year.) 
Who can influence Fred and whom can Fred influence in 
the influence graph in Example 2? 
Construct an influence graph for the board members of a 
company if the President can influence the Director of Re- 
search and Development, the Director of Marketing, and 
the Director of Operations; the Director of Research and 
Development can influence the Director of Operations; 
the Director of Marketing can influence the Director of 
Operations; and no one can influence, or be influenced 
by, the Chief Financial Officer. 
Which other teams did Team 4 beat and which teams beat 
Team 4 in the round-robin tournament represented by the 
graph in Figure 13? 
In around-robin tournament the Tigers beat the Blue Jays, 
the Tigers beat the Cardinals, the Tigers beat the Orioles, 
the Blue Jays beat the Cardinals, the Blue Jays beat the 
Orioles, and the Cardinals beat the Orioles. Model this 
outcome with a directed graph. 
Construct the call graph for a set of seven telephone 
numbers 555-0011, 555-1221, 555-1333, 555-8888, 
555-2222, 555-0091, and 555-1200 if there were three 
calls from 555-0011 to 555-8888 and two calls from 
555-8888 to 555-0011, two calls from 555-2222 to 
555-0091, two calls from 555-1221 to each of the 
other numbers, and one call from 555-1333 to each of 
555-0011, 555-1221, and 555-1200. 
Explain how the two telephone call graphs for calls made 
during the month of January and calls made during the 
month of February can be used to determine the new tele- 
phone numbers of people who have changed their tele- 
phone numbers. 
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a) Explain how graphs can be used to model electronic 
mail messages in a network. Should the edges be di- 
rected or undirected? Should multiple edges be al- 
lowed? Should loops be allowed? 

b) Describe a graph that models the electronic mail sent 
in a network in a particular week. 

How can a graph that models e-mail messages sent in a 
network be used to find people who have recently changed 
their primary e-mail address? 
How can a graph that models e-mail messages sent in 
a network be used to find electronic mail mailing lists 
used to send the same message to many different e-mail 
addresses? 
Describe a graph model that represents whether each per- 
son at a party knows the name of each other person at the 
party. Should the edges be directed or undirected? Should 
multiple edges be allowed? Should loops be allowed? 
Describe a graph model that represents a subway system 
in a large city. Should edges be directed or undirected? 
Should multiple edges be allowed? Should loops be al- 
lowed? 
For each course at a university, there may be one or more 
other courses that are its prerequisites. How can a graph 
be used to model these courses and which courses are pre- 
requisites for which courses? Should edges be directed or 
undirected? Looking at the graph model, how can we find 
courses that do not have any prerequisites and how can 
we find courses that are not the prerequisite for any other 
courses? 

Describe a graph model that represents the positive rec- 

ommendations of movie critics, using vertices to repre- 
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sent both these critics and all movies that are currently 
being shown. 

Describe a graph model that represents traditional mar- 
riages between men and women. Does this graph have 
any special properties? 

Which statements must be executed before S¢ is executed 
in the program in Example 8? (Use the precedence graph 
in Figure 10.) 

Construct a precedence graph for the following program: 


Syixi= 
Soaixi=x4+l1 
S3:y i= 2 
Sa: Z:=y 
S5:x:=x+2 
So: yiex+z 
S7:Z:= 


Describe a discrete structure based on a graph that can 
be used to model airline routes and their flight times. 
[Hint: Add structure to a directed graph. ] 

Describe a discrete structure based on a graph that can be 
used to model relationships between pairs of individuals 
in a group, where each individual may either like, dislike, 
or be neutral about another individual, and the reverse 
relationship may be different. [Hint: Add structure to a 
directed graph. Treat separately the edges in opposite di- 
rections between vertices representing two individuals. ] 

Describe a graph model that can be used to represent all 
forms of electronic communication between two people 
in a single graph. What kind of graph is needed? 


Graph Terminology and Special Types of Graphs 


DEFINITION 1 


Introduction 


We introduce some of the basic vocabulary of graph theory in this section. We will use this vo- 
cabulary later in this chapter when we solve many different types of problems. One such problem 


Links 


involves determining whether a graph can be drawn in the plane so that no two of its edges cross. 
Another example is deciding whether there is a one-to-one correspondence between the vertices 


of two graphs that produces a one-to-one correspondence between the edges of the graphs. We 
will also introduce several important families of graphs often used as examples and in models. 
Several important applications will be described where these special types of graphs arise. 


Basic Terminology 


First, we give some terminology that describes the vertices and edges of undirected graphs. 


Two vertices u and v in an undirected graph G are called adjacent (or neighbors) in G if u 


and v are endpoints of an edge e of G. Such an edge e is called incident with the vertices u 


and v and e is said to connect u and v. 
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DEFINITION 2 


DEFINITION 3 


EXAMPLE 1 


EXAMPLE 2 


We will also find useful terminology describing the set of vertices adjacent to a particular vertex 
of a graph. 


The set of all neighbors of a vertex v of G = (V, E), denoted by N(v), is called the neigh- 
borhood of v. If A is a subset of V, we denote by N(A) the set of all vertices in G that are 
adjacent to at least one vertex in A. So, N(A) = Uc, NW). 


To keep track of how many edges are incident to a vertex, we make the following definition. 


The degree of a vertex in an undirected graph is the number of edges incident with it, except 
that a loop at a vertex contributes twice to the degree of that vertex. The degree of the 
vertex v is denoted by deg(v). 


What are the degrees and what are the neighborhoods of the vertices in the graphs G and H 
displayed in Figure 1? 


Solution: In G, deg(a) = 2, deg(b) = deg(c) = deg(f) = 4, deg(d) = 1, deg(e) = 3, and 
deg(g) = 0. The neighborhoods of these vertices are N(a) = {b, f}, N(b) = {a,c, e, f}, 
N(c) = {b, d,e, f}, N(d) = {c}, N(e) = {b, c, f}, NCS) = {a, b,c, e}, and N(g) =. In 
H, deg(a) = 4, deg(b) = deg(e) = 6, deg(c) = 1, and deg(d) = 5. The neighborhoods of 
these vertices are N(a) = {b, d, e}, N(b) = {a, b,c, d, e}, N(c) = {b}, N(d) = {a, b, e}, and 
N(e) = {a, b, d}. < 


eso 


G H 


FIGURE 1 The Undirected Graphs G and H. 


A vertex of degree zero is called isolated. It follows that an isolated vertex is not adjacent 
to any vertex. Vertex g in graph G in Example | is isolated. A vertex is pendant if and only 
if it has degree one. Consequently, a pendant vertex is adjacent to exactly one other vertex. 
Vertex d in graph G in Example | is pendant. 

Examining the degrees of vertices in a graph model can provide useful information about 
the model, as Example 2 shows. 


What does the degree of a vertex in a niche overlap graph (introduced in Example 11 in Sec- 
tion 10.1) represent? Which vertices in this graph are pendant and which are isolated? Use the 
niche overlap graph shown in Figure 11 of Section 10.1 to interpret your answers. 


Solution: There is an edge between two vertices in a niche overlap graph if and only if the two 
species represented by these vertices compete. Hence, the degree of a vertex in a niche overlap 
graph is the number of species in the ecosystem that compete with the species represented by 
this vertex. A vertex is pendant if the species competes with exactly one other species in the 


THEOREM 1 


EXAMPLE 3 


THEOREM 2 
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ecosystem. Finally, the vertex representing a species is isolated if this species does not compete 
with any other species in the ecosystem. 

For instance, the degree of the vertex representing the squirrel in the niche overlap graph 
in Figure 11 in Section 10.1 is four, because the squirrel competes with four other species: the 
crow, the opossum, the raccoon, and the woodpecker. In this niche overlap graph, the mouse is 
the only species represented by a pendant vertex, because the mouse competes only with the 
shrew and all other species compete with at least two other species. There are no isolated vertices 
in the graph in this niche overlap graph because every species in this ecosystem competes with 
at least one other species. < 


What do we get when we add the degrees of all the vertices of a graph G = (V, E)? Each 
edge contributes two to the sum of the degrees of the vertices because an edge is incident with 
exactly two (possibly equal) vertices. This means that the sum of the degrees of the vertices 
is twice the number of edges. We have the result in Theorem 1, which is sometimes called 
the handshaking theorem (and is also often known as the handshaking lemma), because of the 
analogy between an edge having two endpoints and a handshake involving two hands. 


THE HANDSHAKING THEOREM Let G = (V, E) be an undirected graph with m 


edges. Then 
Lp. = ae deg(v). 
veV 


(Note that this applies even if multiple edges and loops are present.) 


How many edges are there in a graph with 10 vertices each of degree six? 


Solution: Because the sum of the degrees of the vertices is 6 - 10 = 60, it follows that 2m = 60 
where m is the number of edges. Therefore, m = 30. < 


Theorem | shows that the sum of the degrees of the vertices of an undirected graph is even. 
This simple fact has many consequences, one of which is given as Theorem 2. 


An undirected graph has an even number of vertices of odd degree. 


Proof: Let V; and V2 be the set of vertices of even degree and the set of vertices of odd degree, 
respectively, in an undirected graph G = (V, E) with m edges. Then 


2m = > deg(v) = > deg(v) + 2 deg(v). 


veV veVy veV2 


Because deg(v) is even for v € Vj, the first term in the right-hand side of the last equality is 
even. Furthermore, the sum of the two terms on the right-hand side of the last equality is even, 
because this sum is 2m. Hence, the second term in the sum is also even. Because all the terms in 
this sum are odd, there must be an even number of such terms. Thus, there are an even number 
of vertices of odd degree. 4 


Terminology for graphs with directed edges reflects the fact that edges in directed graphs 
have directions. 
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DEFINITION 5 
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When (u, v) is an edge of the graph G with directed edges, u is said to be adjacent to v and v 
is said to be adjacent from u. The vertex u is called the initial vertex of (u, v), and v is called 
the terminal or end vertex of (u, v). The initial vertex and terminal vertex of a loop are the 
same. 


Because the edges in graphs with directed edges are ordered pairs, the definition of the degree 
of a vertex can be refined to reflect the number of edges with this vertex as the initial vertex and 
as the terminal vertex. 


In a graph with directed edges the in-degree of a vertex v, denoted by deg” (v), is the number 
of edges with v as their terminal vertex. The out-degree of v, denoted by deg* (v), is the 
number of edges with v as their initial vertex. (Note that a loop at a vertex contributes | to 
both the in-degree and the out-degree of this vertex.) 


Find the in-degree and out-degree of each vertex in the graph G with directed edges shown in 
Figure 2. 


FIGURE 2 The Directed Graph G. 


Solution: The in-degrees in G are deg” (a) = 2, deg” (b) = 2, deg” (c) = 3, deg” (d) = 2, 
deg (e) = 3, and deg” (f) = 0. The out-degrees are deg* (a) = 4, deg* (b) = 1, deg* (c) = 2, 
deg (d) = 2, deg* (e) = 3, and degt(f) = 0. < 


Because each edge has an initial vertex and a terminal vertex, the sum of the in-degrees and 
the sum of the out-degrees of all vertices in a graph with directed edges are the same. Both of 
these sums are the number of edges in the graph. This result is stated as Theorem 3. 


Let G = (V, E) bea graph with directed edges. Then 


> deg (v) = SS degt (v) = |E|. 


veV veV 


There are many properties of a graph with directed edges that do not depend on the direction 
of its edges. Consequently, it is often useful to ignore these directions. The undirected graph that 
results from ignoring directions of edges is called the underlying undirected graph. A graph 
with directed edges and its underlying undirected graph have the same number of edges. 


Some Special Simple Graphs 


We will now introduce several classes of simple graphs. These graphs are often used as examples 
and arise in many applications. 


EXAMPLE 5 


Ky 
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Complete Graphs A complete graph on n vertices, denoted by Ky, is a simple graph 
that contains exactly one edge between each pair of distinct vertices. The graphs K,, for 
n = 1,2,3,4,5, 6, are displayed in Figure 3. A simple graph for which there is at least one 
pair of distinct vertex not connected by an edge is called noncomplete. < 


Ko 


FIGURE3 The Graphs K,, for 1 < n < 6. 


EXAMPLE 6 


EXAMPLE 7 


EXAMPLE 8 


Cycles A cycle C,, n > 3, consists of n vertices v1, v2,...,V, and edges {v1, v2}, 
{v2, V3},---,{Vn—1, Vn}, and {v,, vy}. The cycles C3, C4, C5, and Ce are displayed in 
Figure 4. < 


ANU © 


C 


FIGURE 4 The Cycles C3, C4, Cs, and Cg. 


Wheels We obtain a wheel W,, when we add an additional vertex to a cycle Cy, for n > 3, 
and connect this new vertex to each of the n vertices in C,,, by new edges. The wheels W3, W4, 
Ws, and Wg are displayed in Figure 5. 


AK & & 


FIGURE 5 The Wheels W3, W4, Ws, and We. 


n-Cubes Ann-dimensional hypercube, or -cube, denoted by Q», is a graph that has vertices 
representing the 2” bit strings of length n. Two vertices are adjacent if and only if the bit strings 
that they represent differ in exactly one bit position. We display Q1, Q2, and Q3 in Figure 6. 
Note that you can construct the (n + 1)-cube Q,,4; from the n-cube Q, by making two 
copies of Q,, prefacing the labels on the vertices with a 0 in one copy of Q,, and with a | in the 
other copy of Q,, and adding edges connecting two vertices that have labels differing only in 
the first bit. In Figure 6, Q3 is constructed from Q2 by drawing two copies of Qo as the top and 
bottom faces of Q3, adding 0 at the beginning of the label of each vertex in the bottom face and 
1 at the beginning of the label of each vertex in the top face. (Here, by face we mean a face of 
a cube in three-dimensional space. Think of drawing the graph Q3 in three-dimensional space 
with copies of Q2 as the top and bottom faces of a cube and then drawing the projection of the 
resulting depiction in the plane.) < 
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Q1 Q> Q; 


FIGURE 6 Then-cube Q,, n = 1, 2, 3. 


Bipartite Graphs 


Sometimes a graph has the property that its vertex set can be divided into two disjoint subsets 
such that each edge connects a vertex in one of these subsets to a vertex in the other subset. 
For example, consider the graph representing marriages between men and women in a village, 
where each person is represented by a vertex and a marriage is represented by an edge. In this 
graph, each edge connects a vertex in the subset of vertices representing males and a vertex in 
the subset of vertices representing females. This leads us to Definition 5. 


A simple graph G is called bipartite if its vertex set V can be partitioned into two disjoint 
sets V; and V2 such that every edge in the graph connects a vertex in V; and a vertex in V2 
(so that no edge in G connects either two vertices in V; or two vertices in V2). When this 
condition holds, we call the pair (V,, V2) a bipartition of the vertex set V of G. 


In Example 9 we will show that C¢ is bipartite, and in Example 10 we will show that K3 is 
not bipartite. 


C¢ 1s bipartite, as shown in Figure 7, because its vertex set can be partitioned into the two sets 
Vi = {v1, v3, v5} and V2 = {vo, va, ve}, and every edge of Ce connects a vertex in V; and a 
vertex in V2. 


K3 is not bipartite. To verify this, note that if we divide the vertex set of K3 into two disjoint 
sets, one of the two sets must contain two vertices. If the graph were bipartite, these two vertices 
could not be connected by an edge, but in K3 each vertex is connected to every other vertex by 
an edge. < 


Are the graphs G and H displayed in Figure 8 bipartite? 


a b a b 
# c 
f ¢ 
f 
e d e d 


FIGURE 7 Showing That C¢ Is 


Bipartite. 


G H 


FIGURE 8 The Undirected Graphs G and H. 
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Solution: Graph G is bipartite because its vertex set is the union of two disjoint sets, {a, b, d} 
and {c, e, f, g}, and each edge connects a vertex in one of these subsets to a vertex in the other 
subset. (Note that for G to be bipartite it is not necessary that every vertex in {a, b, d} be adjacent 
to every vertex in {c, e, f, g}. For instance, b and g are not adjacent.) 

Graph H is not bipartite because its vertex set cannot be partitioned into two subsets so 
that edges do not connect two vertices from the same subset. (The reader should verify this by 
considering the vertices a, b, and f.) < 


Theorem 4 provides a useful criterion for determining whether a graph is bipartite. 


A simple graph is bipartite if and only if it is possible to assign one of two different colors to 
each vertex of the graph so that no two adjacent vertices are assigned the same color. 


Proof: First, suppose that G = (V, £) is a bipartite simple graph. Then V = V; U V2, where V, 
and V2 are disjoint sets and every edge in E connects a vertex in V; and a vertex in V2. If we 
assign one color to each vertex in V; and a second color to each vertex in V2, then no two 
adjacent vertices are assigned the same color. 

Now suppose that it is possible to assign colors to the vertices of the graph using just two 
colors so that no two adjacent vertices are assigned the same color. Let V; be the set of vertices 
assigned one color and V2 be the set of vertices assigned the other color. Then, V; and V2 
are disjoint and V = V; U Vo. Furthermore, every edge connects a vertex in V; and a vertex 
in V2 because no two adjacent vertices are either both in V; or both in V2. Consequently, G 
is bipartite. 


We illustrate how Theorem 4 can be used to determine whether a graph is bipartite in 
Example 12. 


Use Theorem 4 to determine whether the graphs in Example 11 are bipartite. 


Solution: We first consider the graph G. We will try to assign one of two colors, say red and 
blue, to each vertex in G so that no edge in G connects a red vertex and a blue vertex. Without 
loss of generality we begin by arbitrarily assigning red to a. Then, we must assign blue to c, e, 
f,and g, because each of these vertices is adjacent to a. To avoid having an edge with two blue 
endpoints, we must assign red to all the vertices adjacent to either c, e, f, or g. This means that 
we must assign red to both b and d (and means that a must be assigned red, which it already has 
been). We have now assigned colors to all vertices, with a, b, and d red and c, e, f, and g blue. 
Checking all edges, we see that every edge connects a red vertex and a blue vertex. Hence, by 
Theorem 4 the graph G is bipartite. 

Next, we will try to assign either red or blue to each vertex in H so that no edge in H 
connects a red vertex and a blue vertex. Without loss of generality we arbitrarily assign red to a. 
Then, we must assign blue to b, e, and f, because each is adjacent to a. But this is not possible 
because e and f are adjacent, so both cannot be assigned blue. This argument shows that we 
cannot assign one of two colors to each of the vertices of H so that no adjacent vertices are 
assigned the same color. It follows by Theorem 4 that H is not bipartite. < 


Theorem 4 is an example of a result in the part of graph theory known as graph colorings. 
Graph colorings is an important part of graph theory with important applications. We will study 
graph colorings further in Section 10.8. 

Another useful criterion for determining whether a graph is bipartite is based on the notion 
of a path, a topic we study in Section 10.4. A graph is bipartite if and only if it is not possible 
to start at a vertex and return to this vertex by traversing an odd number of distinct edges. We 
will make this notion more precise when we discuss paths and circuits in graphs in Section 10.4 
(see Exercise 63 in that section). 
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Complete Bipartite Graphs A complete bipartite graph K,,,n is a graph that has its vertex 
set partitioned into two subsets of m and n vertices, respectively with an edge between two 
vertices if and only if one vertex is in the first subset and the other vertex is in the second subset. 
The complete bipartite graphs K2,3, K3,3, K3,5, and K2.6 are displayed in Figure 9. < 


A 


2,3 3,3 


35 2,6 


FIGURE 9 Some Complete Bipartite Graphs. 


Bipartite Graphs and Matchings 


Bipartite graphs can be used to model many types of applications that involve matching the 
elements of one set to elements of another, as Example 14 illustrates. 


Job Assignments Suppose that there are m employees in a group and n different jobs that 
need to be done, where m > n. Each employee is trained to do one or more of these n jobs. We 
would like to assign an employee to each job. To help with this task, we can use a graph to model 
employee capabilities. We represent each employee by a vertex and each job by a vertex. For 
each employee, we include an edge from that employee to all jobs that the employee has been 
trained to do. Note that the vertex set of this graph can be partitioned into two disjoint sets, the 
set of employees and the set of jobs, and each edge connects an employee to a job. Consequently, 
this graph is bipartite, where the bipartition is (EZ, J) where E is the set of employees and J is 
the set of jobs. We now consider two different scenarios. 

First, suppose that a group has four employees: Alvarez, Berkowitz, Chen, and Davis; 
and suppose that four jobs need to be done to complete Project 1: requirements, architecture, 
implementation, and testing. Suppose that Alvarez has been trained to do requirements and 
testing; Berkowitz has been trained to do architecture, implementation, and testing; Chen has 
been trained to do requirements, architecture, and implementation; and Davis has only been 
trained to do requirements. We model these employee capabilities using the bipartite graph in 
Figure 10(a). 

Second, suppose that a group has second group also has four employees: Washington, Xuan, 
Ybarra, and Ziegler; and suppose that the same four jobs need to be done to complete Project 2 as 
are needed to complete Project 1. Suppose that Washington has been trained to do architecture; 
Xuan has been trained to do requirements, implementation, and testing; Ybarra has been trained 
to do architecture; and Ziegler has been trained to do requirements, architecture and testing. We 
model these employee capabilities using the bipartite graph in Figure 10(b). 

To complete Project 1, we must assign an employee to each job so that every job has an 
employee assigned to it, and so that no employee is assigned more than one job. We can do this 
by assigning Alvarez to testing, Berkowitz to implementation, Chen to architecture, and Davis 
to requirements, as shown in Figure 10(a) (where blue lines show this assignment of jobs). 

To complete Project 2, we must also assign an employee to each job so that every job has 
an employee assigned to it and no employee is assigned more than one job. However, this is 
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Hall’s marriage theorem 
is an example of a 
theorem where obvious 
necessary conditions are 
sufficient too. 
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Alvarez _ Berkowitz Chen Davis Washington Xuan Ybarra Ziegler 
requirements architecture implementation testing requirements architecture implementation testing 
(a) (b) 


FIGURE 10 Modeling the Jobs for Which Employees Have Been Trained. 


impossible because there are only two employees, Xuan and Ziegler, who have been trained for 
at least one of the three jobs of requirements, implementation, and testing. Consequently, there 
is no way to assign three different employees to these three job so that each job is assigned an 
employee with the appropriate training. < 


Finding an assignment of jobs to employees can be thought of as finding a matching in the 
graph model, where a matching M in a simple graph G = (V, E) is a subset of the set E of 
edges of the graph such that no two edges are incident with the same vertex. In other words, a 
matching is a subset of edges such that if {s, t} and {u, v} are distinct edges of the matching, 
then s, t, wu, and v are distinct. A vertex that is the endpoint of an edge of a matching M is said to 
be matched in M; otherwise it is said to be unmatched. A maximum matching is a matching 
with the largest number of edges. We say that a matching M in a bipartite graph G = (V, E) 
with bipartition (V;, V2) is a complete matching from V, to V2 if every vertex in Vj is the 
endpoint of an edge in the matching, or equivalently, if |M| = |Vi|. For example, to assign jobs 
to employees so that the largest number of jobs are assigned employees, we seek a maximum 
matching in the graph that models employee capabilities. To assign employees to all jobs we 
seek a complete matching from the set of jobs to the set of employees. In Example 14, we found 
a complete matching from the set of jobs to the set of employees for Project 1, and this matching 
is a maximun matching, and we showed that no complete matching exists from the set of jobs 
to the employees for Project 2. 

We now give an example of how matchings can be used to model marriages. 


Marriages onanIsland Suppose that there are m men andn women on an island. Each person 
has a list of members of the opposite gender acceptable as a spouse. We construct a bipartite 
graph G = (V\, V2) where V; is the set of men and V2 is the set of women so that there is an 
edge between a man and a woman if they find each other acceptable as a spouse. A matching in 
this graph consists of a set of edges, where each pair of endpoints of an edge is a husband-wife 
pair. A maximum matching is a largest possible set of married couples, and a complete matching 
of V| is a set of married couples where every man is married, but possibly not all women. <4 


NECESSARY AND SUFFICIENT CONDITIONS FOR COMPLETE MATCHINGS We 
now turn our attention to the question of determining whether a complete matching from V; 
to V2 exists when (V}, V2) is a bipartition of a bipartite graph G = (V, E). We will introduce a 
theorem that provides a set of necessary and sufficient conditions for the existence of a complete 
matching. This theorem was proved by Philip Hall in 1935. 


HALL’S MARRIAGE THEOREM The bipartite graph G = (V, E) with bipartition 
(V1, V2) has a complete matching from V, to V2 if and only if |N(A)| > |A| for all subsets 
A of Vi i 


660 


10 / Graphs 


Proof: We first prove the only if part of the theorem. To do so, suppose that there is a complete 
matching M from V; to V2. Then, if A C Vj, for every vertex v € A, there is an edge in M@ 
connecting v to a vertex in V2. Consequently, there are at least as many vertices in V2 that are 
neighbors of vertices in V; as there are vertices in V. It follows that |N(A)| > |A|. 

To prove the if part of the theorem, the more difficult part, we need to show that if 
|N(A)| => |A| for all A C Vj, then there is a complete matching M from V, to V2. We will 
use strong induction on | V;| to prove this. 


Basis step: If |Vi| = 1, then V, contains a single vertex vo. Because |N({vo})| = |{vo}] = 1, 
there is at least one edge connecting vg and a vertex wo € V2. Any such edge forms a complete 
matching from Vj to V2. 


Inductive step: We first state the inductive hypothesis. 


Inductive hypothesis: Let k be a positive integer. If G = (V, E) is a bipartite graph with bipar- 
tition (V;, V2), and |V\| = j < k, then there is a complete matching M from V to V2 whenever 
the condition that |N(A)| > |A| for all A C Vj is met. 

Now suppose that H = (W, F) is a bipartite graph with bipartition (W;, W2) and |W;| = 
k + 1. We will prove that the inductive holds using a proof by cases, using two case. Case (i) 
applies when for all integers 7 with | < 7 < k, the vertices in every set of j elements from W are 
adjacent to at least j + 1 elements of W2. Case (ii) applies when for some j with | < j < k there 
is a subset W; of j vertices such that there are exactly j neighbors of these vertices in W2. Because 
either Case (i) or Case (ii) holds, we need only consider these cases to complete the inductive step. 


Case (i): Suppose that for all integers j with 1 < j < k, the vertices in every subset of j elements 
from W are adjacent to at least 7 + | elements of W2. Then, we select a vertex v € W; and an 
element w € N({v}), which must exist by our assumption that |N({v}| > |{v}| = 1. We delete 
v and w and all edges incident to them from H. This produces a bipartite graph H’ with 
bipartition (W, — {v}, W2 — {w}). Because |W; — {v}| =k, the inductive hypothesis tells us 
there is a complete matching from W, — {v} to W2 — {w}. Adding the edge from v to w to this 
complete matching produces a complete matching from W; to Wo. 


Case (ii): Suppose that for some j with 1 < j <k, there is a subset W; of j vertices such that 
there are exactly j neighbors of these vertices in W2. Let W; be the set of these neighbors. Then, 
by the inductive hypothesis there is a complete matching from W; to W5. Remove these 2j 
vertices from W, and W) and all incident edges to produce a bipartite graph K with bipartition 
(Wi — Wi, Wo — W5). 

We will show that the graph K satisfies the condition |N(A)| > |A| for all subsets A of 
Wi - Wj. If not, there would be a subset of ¢ vertices of W, — Wi where | <t<k+1-j 
such that the vertices in this subset have fewer than f vertices of W2 — Ws as neighbors. Then, 
the set of j +f vertices of W, consisting of these ¢ vertices together with the j vertices we 
removed from W, has fewer than j +¢ neighbors in W2, contradicting the hypothesis that 
|N(A)| = |A| for all A C Wy. 


PHILIP HALL (1904-1982) Philip Hall grew up in London, where his mother was a dressmaker. He won a 
scholarship for board school reserved for needy children, and later a scholarship to King’s College of Cambridge 
University. He received his bachelors degree in 1925. In 1926, unsure of his career goals, he took a civil service 
exam, but decided to continue his studies at Cambridge after failing. 

In 1927 Hall was elected to a fellowship at King’s College; soon after, he made his first important discovery 
in group theory. The results he proved are now known as Hall’s theorems. In 1933 he was appointed as a Lecturer 
at Cambridge, where he remained until 1941. During World War II he worked as a cryptographer at Bletchley 
Park breaking Italian and Japanese codes. At the end of the war, Hall returned to King’s College, and was soon 
promoted. In 1953 he was appointed to the Sadleirian Chair. His work during the 1950s proved to be extremely 


influential to the rapid development of group theory during the 1960s. 


Hall loved poetry and recited it beautifully in Italian and Japanese, as well as English. He was interested in art, music, and 


botany. He was quite shy and disliked large groups of people. Hall had an incredibly broad and varied knowledge, and was respected 
for his integrity, intellectual standards, and judgement. He was beloved by his students. 
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Hence, by the inductive hypothesis, the graph K has a complete matching. Combining this 
complete matching with the complete matching from W; to W;, we obtain a complete matching 
from W, to Wp. 


We have shown that in both cases there is a complete matching from W; to W2. This 
completes the inductive step and completes the proof. | 


We have used strong induction to prove Hall’s marriage theorem. Although our proof is 
elegant, it does have some drawbacks. In particular, we cannot construct an algorithm based on 
this proof that finds a complete matching in a bipartite graph. For a constructive proof that can 
be used as the basis of an algorithm, see [Gi85]. 


Some Applications of Special Types of Graphs 


We conclude this section by introducing some additional graph models that involve the special 
types of graph we have discussed in this section. 


Local Area Networks The various computers in a building, such as minicomputers and per- 
sonal computers, as well as peripheral devices such as printers and plotters, can be connected 
using a local area network. Some of these networks are based on a star topology, where all 
devices are connected to a central control device. A local area network can be represented using 
a complete bipartite graph K1,,, as shown in Figure 11(a). Messages are sent from device to 
device through the central control device. 


(a) (b) (c) 


FIGURE 11 Star, Ring, and Hybrid Topologies for Local Area Networks. 


Other local area networks are based on a ring topology, where each device is connected to 
exactly two others. Local area networks with a ring topology are modeled using n-cycles, Cy, 
as shown in Figure 11(b). Messages are sent from device to device around the cycle until the 
intended recipient of a message is reached. 

Finally, some local area networks use a hybrid of these two topologies. Messages may be 
sent around the ring, or through a central device. This redundancy makes the network more 
reliable. Local area networks with this redundancy can be modeled using wheels W,,, as shown 
in Figure 11(c). < 


Interconnection Networks for Parallel Computation For many years, computers executed 
programs one operation at a time. Consequently, the algorithms written to solve problems were 
designed to perform one step at a time; such algorithms are called serial. (Almost all algorithms 
described in this book are serial.) However, many computationally intense problems, such as 
weather simulations, medical imaging, and cryptanalysis, cannot be solved in a reasonable 
amount of time using serial operations, even on a supercomputer. Furthermore, there is a physical 
limit to how fast a computer can carry out basic operations, so there will always be problems 
that cannot be solved in a reasonable length of time using serial operations. 

Parallel processing, which uses computers made up of many separate processors, each 
with its own memory, helps overcome the limitations of computers with a single processor. 
Parallel algorithms, which break a problem into a number of subproblems that can be solved 
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concurrently, can then be devised to rapidly solve problems using a computer with multiple 
processors. In a parallel algorithm, a single instruction stream controls the execution of the 
algorithm, sending subproblems to different processors, and directs the input and output of 
these subproblems to the appropriate processors. 

When parallel processing is used, one processor may need output generated by another 
processor. Consequently, these processors need to be interconnected. We can use the appropriate 
type of graph to represent the interconnection network of the processors in a computer with 
multiple processors. In the following discussion, we will describe the most commonly used 
types of interconnection networks for parallel processors. The type of interconnection network 
used to implement a particular parallel algorithm depends on the requirements for exchange of 
data between processors, the desired speed, and, of course, the available hardware. 

The simplest, but most expensive, network-interconnecting processors include a two-way 
link between each pair of processors. This network can be represented by K,,, the complete 
graph on n vertices, when there are n processors. However, there are serious problems with 
this type of interconnection network because the required number of connections is so large. 
In reality, the number of direct connections to a processor is limited, so when there are a large 
number of processors, a processor cannot be linked directly to all others. For example, when 
there are 64 processors, C(64, 2) = 2016 connections would be required, and each processor 
would have to be directly connected to 63 others. 

On the other hand, perhaps the simplest way to interconnect n processors is to use an 
arrangement known as a linear array. Each processor P;, other than P; and P,, is connected to 
its neighbors P;_; and Pj; via a two-way link. P; is connected only to P2, and P,, is connected 
only to P,—1. The linear array for six processors is shown in Figure 12. The advantage of a 
linear array is that each processor has at most two direct connections to other processors. The 
disadvantage is that it is sometimes necessary to use a large number of intermediate links, called 
hops, for processors to share information. 

The mesh network (or two-dimensional array) is a commonly used interconnection net- 
work. In such a network, the number of processors is a perfect square, say n = m?. The n 
processors are labeled P(i, j),0 <i <m-—=1,0< j <m -— 1. Two-way links connect proces- 
sor P(i, j) with its four neighbors, processors P(i +1, j) and P(i, j +1), as long as these 
are processors in the mesh. (Note that four processors, on the corners of the mesh, have only 
two adjacent processors, and other processors on the boundaries have only three neighbors. 
Sometimes a variant of a mesh network in which every processor has exactly four connections 
is used; see Exercise 72.) The mesh network limits the number of links for each processor. Com- 
munication between some pairs of processors requires O(./n) = O(m) intermediate links. (See 
Exercise 73.) The graph representing the mesh network for 16 processors is shown in Figure 13. 

One important type of interconnection network is the hypercube. For such a network, the 
number of processors is a power of 2, n = 2”. The n processors are labeled Po, Pj, ..., Pn—1- 
Each processor has two-way connections to m other processors. Processor P; is linked to the 
processors with indices whose binary representations differ from the binary representation of i 


DEFINITION 7 


DEFINITION 8 


Po 


EXAMPLE 18 
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in exactly one bit. The hypercube network balances the number of direct connections for each 
processor and the number of intermediate connections required so that processors can com- 
municate. Many computers have been built using a hypercube network, and many parallel 
algorithms have been devised that use a hypercube network. The graph Q,,, the m-cube, rep- 
resents the hypercube network with n = 2” processors. Figure 14 displays the hypercube net- 
work for eight processors. (Figure 14 displays a different way to draw Q3 than was shown in 
Figure 6.) < 


New Graphs from Old 


Sometimes we need only part of a graph to solve a problem. For instance, we may care only 
about the part of a large computer network that involves the computer centers in New York, 
Denver, Detroit, and Atlanta. Then we can ignore the other computer centers and all telephone 
lines not linking two of these specific four computer centers. In the graph model for the large 
network, we can remove the vertices corresponding to the computer centers other than the four 
of interest, and we can remove all edges incident with a vertex that was removed. When edges 
and vertices are removed from a graph, without removing endpoints of any remaining edges, a 
smaller graph is obtained. Such a graph is called a subgraph of the original graph. 


A subgraph of a graph G = (V, E) is a graph H = (W, F), where W C V and FC E.A 
subgraph H of G is a proper subgraph of G if H € G. 


Given a set of vertices of a graph, we can form a subgraph of this graph with these vertices 
and the edges of the graph that connect them. 


Let G = (V, E) beasimple graph. The subgraph induced by a subset W of the vertex set V 
is the graph (W, F), where the edge set F contains an edge in E if and only if both endpoints 
of this edge are in W. 


The graph G shown in Figure 15 is a subgraph of Ks. If we add the edge connecting c and e to 
G, we obtain the subgraph induced by W = {a, b,c, e}. < 


REMOVING OR ADDING EDGES OF A GRAPH_ Given a graph G = (V, E) and an edge 
e € E, wecan produce a subgraph of G by removing the edge e. The resulting subgraph, denoted 
by G — e, has the same vertex set V as G. Its edge set is E — e. Hence, 


G—e=(V, E—{e}). 


Similarly, if E’ is a subset of E, we can produce a subgraph of G by removing the edges in E’ 
from the graph. The resulting subgraph has the same vertex set V as G. Its edge set is E — E’. 


P, P, Ps Pe P, 


a —e. . d € c 


FIGURE 14 A Hypercube Network for Eight Processors. 


FIGURE 15 A Subgraph of Ks. 
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FIGURE 16 (a) The Simple Graphs G, and G2; (b) Their Union G, U G2. 


We can also add an edge e to a graph to produce a new larger graph when this edge connects 
two vertices already in G. We denote by G + e the new graph produced by adding a new edge e, 
connecting two previously nonincident vertices, to the graph G Hence, 


Gte=(V, EU {ep. 


The vertex set of G + e is the same as the vertex set of G and the edge set is the union of the 
edge set of G and the set {e}. 


EDGE CONTRACTIONS Sometimes when we remove an edge from a graph, we do not 
want to retain the endpoints of this edge as separate vertices in the resulting subgraph. In such 
a case we perform an edge contraction which removes an edge e with endpoints wu and v and 
merges u and w into a new single vertex w, and for each edge with u or v as an endpoint replaces 
the edge with one with w as endpoint in place of u or v and with the same second endpoint. 
Hence, the contraction of the edge e with endpoints u and v in the graph G = (V, E) produces a 
new graph G’ = (V’, E’) (which is not a subgraph of G), where V’ = V — {u, v} U {w} and E’ 
contains the edges in E which do not have either u or v as endpoints and an edge connecting w 
to every neighbor of either u or v in V. For example, the contraction of the edge connecting the 
vertices e and c in the graph G, in Figure 16 produces a new graph G‘, with vertices a, b, d, 
and w. As in Gj, there is an edge in G‘, connecting a and b and an edge connecting a and d. 
There also is an edge in G/ that connects b and w that replaces the edges connecting b and c and 
connecting b and e in G; and an edge in G’, that connects d and w replacing the edge connecting 
dande in G,. 


REMOVING VERTICES FROM A GRAPH When we remove a vertex v and all edges 
incident to it from G = (V, E), we produce a subgraph, denoted by G — v. Observe that 
G-—v=(V—v, E’), where E’ is the set of edges of G not incident to v. Similarly, if V’ 
is a subset of V, then the graph G — V’ is the subgraph (V — V’, E’), where E’ is the set of 
edges of G not incident to a vertex in V’. 


GRAPH UNIONS Two or more graphs can be combined in various ways. The new graph that 


contains all the vertices and edges of these graphs is called the union of the graphs. We will 
give a more formal definition for the union of two simple graphs. 


The union of two simple graphs G; = (V;, E,) and G2 = (V2, E2) is the simple graph with 
vertex set V; U V2 and edge set FE, U E>. The union of G, and G2 is denoted by G; U Go. 


Find the union of the graphs G; and G2 shown in Figure 16(a). < 
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Solution: The vertex set of the union G; U G2 is the union of the two vertex sets, namely, 
{a, b,c, d,e, f}. The edge set of the union is the union of the two edge sets. The union is 


displayed in Figure 16(b). 


Exercises 


In Exercises 1-3 find the number of vertices, the number of 
edges, and the degree of each vertex in the given undirected 
graph. Identify all isolated and pendant vertices. 


1, 2 b c 
VA | 
e 
f e d 
2. a b 
VN 
e d 
f i h g e 
4. Find the sum of the degrees of the vertices of each graph 


in Exercises 1—3 and verify that it equals twice the number 
of edges in the graph. 


5. Can a simple graph exist with 15 vertices each of degree 
five? 


6. Show that the sum, over the set of people at a party, of 
the number of people a person has shaken hands with, is 
even. Assume that no one shakes his or her own hand. 


In Exercises 7—9 determine the number of vertices and edges 
and find the in-degree and out-degree of each vertex for the 
given directed multigraph. 


7. b 8. a b 


10. For each of the graphs in Exercises 7—9 determine the 
sum of the in-degrees of the vertices and the sum of the 
out-degrees of the vertices directly. Show that they are 
both equal to the number of edges in the graph. 

11. Construct the underlying undirected graph for the graph 
with directed edges in Figure 2. 

12. What does the degree of a vertex represent in the acquain- 
tanceship graph, where vertices represent all the people 
in the world? What does the neighborhood a vertex in this 
graph represent? What do isolated and pendant vertices 
in this graph represent? In one study it was estimated that 
the average degree of a vertex in this graph is 1000. What 
does this mean in terms of the model? 

13. What does the degree of a vertex represent in an academic 
collaboration graph? What does the neighborhood of a 
vertex represent? What do isolated and pendant vertices 
represent? 

14. What does the degree of a vertex in the Hollywood graph 
represent? What does the neighborhood of a vertex repre- 
sent? What do the isolated and pendant vertices represent? 

15. What do the in-degree and the out-degree of a vertex in a 
telephone call graph, as described in Example 4 of Sec- 
tion 10.1, represent? What does the degree of a vertex in 
the undirected version of this graph represent? 

16. What do the in-degree and the out-degree of a vertex in 
the Web graph, as described in Example 5 of Section 10.1, 
represent? 

17. What do the in-degree and the out-degree of a vertex in a 
directed graph modeling a round-robin tournament rep- 
resent? 

18. Show that in a simple graph with at least two vertices 
there must be two vertices that have the same degree. 

19. Use Exercise 18 to show that in a group of people, there 
must be two people who are friends with the same number 
of other people in the group. 

20. Draw these graphs. 

a) Ky b) Ki.8 c) K44 
d) C e) Wr f) Q4 

In Exercises 21-25 determine whether the graph is bipartite. 

You may find it useful to apply Theorem 4 and answer the 

question by determining whether it is possible to assign either 

red or blue to each vertex so that no two adjacent vertices are 
assigned the same color. 

21. a b 22. b é 
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é 
For which values of n are these graphs bipartite? 
a) Kn b) Ch c) Wh d) Qn 
Suppose that there are four employees in the computer 
support group of the School of Engineering of a large 
university. Each employee will be assigned to support 
one of four different areas: hardware, software, network- 
ing, and wireless. Suppose that Ping is qualified to support 
hardware, networking, and wireless; Quiggley is qualified 
to support software and networking; Ruiz is qualified to 
support networking and wireless, and Sitea is qualified to 
support hardware and software. 


a) Usea bipartite graph to model the four employees and 
their qualifications. 

b) Use Hall’s theorem to determine whether there is an 
assignment of employees to support areas so that each 
employee is assigned one area to support. 

c) Ifan assignment of employees to support areas so that 
each employee is assigned to one support area exists, 
find one. 


Suppose that a new company has five employees: Zamora, 
Agraharam, Smith, Chou, and Macintyre. Each employee 
will assume one of six responsiblities: planning, public- 
ity, sales, marketing, development, and industry relations. 
Each employee is capable of doing one or more of these 
jobs: Zamora could do planning, sales, marketing, or in- 
dustry relations; Agraharam could do planning or devel- 
opment; Smith could do publicity, sales, or industry re- 
lations; Chou could do planning, sales, or industry rela- 
tions; and Macintyre could do planning, publicity, sales, 
or industry relations. 
a) Model the capabilities of these employees using a bi- 
partite graph. 
b) Find an assignment of responsibilites such that each 
employee is assigned one responsibility. 


29. 


30. 


*31. 


* 32. 


33. 


34. 


c) Is the matching of responsibilities you found in part 
(b) a complete matching? Is it a maximum matching? 


Suppose that there are five young women and five young 
men on an island. Each man is willing to marry some of 
the women on the island and each woman is willing to 
marry any man who is willing to marry her. Suppose that 
Sandeep is willing to marry Tina and Vandana; Barry is 
willing to marry Tina, Xia, and Uma; Teja is willing to 
marry Tina and Zelda; Anil is willing to marry Vandana 
and Zelda; and Emilio is willing to marry Tina and Zelda. 
Use Hall’s theorem to show there is no matching of the 
young men and young women on the island such that each 
young man is matched with a young woman he is willing 
to marry. 


Suppose that there are five young women and six young 
men on an island. Each woman is willing to marry some 
of the men on the island and each man is willing to marry 
any woman who is willing to marry him. Suppose that 
Anna is willing to marry Jason, Larry, and Matt; Barbara 
is willing to marry Kevin and Larry; Carol is willing to 
marry Jason, Nick, and Oscar; Diane is willing to marry 
Jason, Larry, Nick, and Oscar; and Elizabeth is willing to 
marry Jason and Matt. 


a) Model the possible marriages on the island using a 
bipartite graph. 

b) Find a matching of the young women and the young 
men on the island such that each young woman is 
matched with a young man whom she is willing to 
marry. 

c) Is the matching you found in part (b) a complete 
matching? Is it a maximum matching? 


Suppose there is an integer k such that every man on a 
desert island is willing to marry exactly k of the women 
on the island and every woman on the island is willing to 
marry exactly k of the men. Also, suppose that a man is 
willing to marry a woman if and only if she is willing to 
marry him. Show that it is possible to match the men and 
women on the island so that everyone is matched with 
someone that they are willing to marry. 


In this exercise we prove a theorem of @ystein Ore. 
Suppose that G = (V, E) is a bipartite graph with bipar- 
tition (V;, V2) and that A C V;. Show that the maximum 
number of vertices of V; that are the endpoints of a 
matching of G equals |V;| — max,jcy,def(A), where 
def(A) = |A| — |N(A)|. (Here, def(A) is called the de- 
ficiency of A.) [Hint: Form a larger graph by adding 
max ,cy,def(A) new vertices to V2 and connect all of 
them to the vertices of V;.] 


For the graph G in Exercise 1 find 

a) the subgraph induced by the vertices a, b, c, and f. 

b) the new graph G, obtained from G by contracting the 
edge connecting b and f. 


Let n be a positive integer. Show that a subgraph induced 
by anonempty subset of the vertex set of K,, is acomplete 
graph. 


35. How many vertices and how many edges do these graphs 


have? 
a) Ky b) C, c) Wh 
d) Kinn e) Qn 


The degree sequence of a graph is the sequence of the de- 

grees of the vertices of the graph in nonincreasing order. For 

example, the degree sequence of the graph G in Example | is 

4,4,4, 3,2, 1,0. 

36. Find the degree sequences for each of the graphs in 
Exercises 21-25. 

37. Find the degree sequence of each of the following 


graphs. 
a) K4 b) C4 c) W4 
d) K23 e) Q3 


38. What is the degree sequence of the bipartite graph Kj, ,n 
where m and v are positive integers? Explain your answer. 

39. What is the degree sequence of K,, where n is a positive 
integer? Explain your answer. 

40. How many edges does a graph have if its degree sequence 
is 4, 3, 3, 2, 2? Draw such a graph. 

41. How many edges does a graph have if its degree sequence 
is 5,2, 2, 2, 2, 1? Draw such a graph. 

A sequence dj, dz, ..., dy is called graphic if it is the degree 

sequence of a simple graph. 

42. Determine whether each of these sequences is graphic. 
For those that are, draw a graph having the given degree 
sequence. 

a) 5,4,3,2,1,0 b) 6,5,4,3,2,1 ©) 
d) 3,3,3,2,2,2 e) 3,3,2,2,2,2 f) 
g) 5,3,3,3,3,3 h) 5,5, 4,3, 2,1 

43. Determine whether each of these sequences is graphic. 
For those that are, draw a graph having the given degree 
sequence. 

a) 3,3,3,3,2  b) 5,4,3 
d) 4, 4, 3, 3,3 e) 3,2,2,1,0 f) 1L1,1,1,1 
*44, Suppose that d1, d2,..., dy is a graphic sequence. Show 


2, 2;2;2,.2; 2 
1,1,1,1,1,1 


gts ts 


that there is a simple graph with vertices v1, v2,..., Vy 
such that deg(v;) = d; fori = 1,2,...,n and vy is adja- 
cent to v2,..., Vaj41- 

* 45. Show that a sequence dj, do, ..., d, of nonnegative inte- 


gers in nonincreasing order is a graphic sequence if and 
only if the sequence obtained by reordering the terms 


of the sequence dz — 1,..., daj41 — 1, da,42,..., dn So 
that the terms are in nonincreasing order is a graphic 
sequence. 


*46. Use Exercise 45 to construct a recursive algorithm for de- 
termining whether a nonincreasing sequence of positive 
integers is graphic. 

47. Show that every nonincreasing sequence of nonnegative 
integers with an even sum of its terms is the degree se- 
quence of a pseudograph, that is, an undirected graph 
where loops are allowed. [Hint: Construct such a graph 
by first adding as many loops as possible at each vertex. 
Then add additional edges connecting vertices of odd 
degree. Explain why this construction works. ] 

48. How many subgraphs with at least one vertex does K2 
have? 
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49. How many subgraphs with at least one vertex does K3 
have? 


50. How many subgraphs with at least one vertex does W3 
have? 


51. Draw all subgraphs of this graph. 


a b 


c d 


52. Let G be a graph with v vertices and e edges. Let M be 
the maximum degree of the vertices of G, and let m be 
the minimum degree of the vertices of G. Show that 
a) 2e/v>m. b) 2e/v < M. 

A simple graph is called regular if every vertex of this graph 

has the same degree. A regular graph is called n-regular if 

every vertex in this graph has degree n. 

53. For which values of n are these graphs regular? 

a) Kn b) Ch c) Wr d) Qn 

54. For which values of m and n is Ky,» regular? 

55. How many vertices does a regular graph of degree four 
with 10 edges have? 

In Exercises 56-58 find the union of the given pair of simple 

graphs. (Assume edges with the same endpoints are the same.) 


56. a 


a b 
; b 
e fa 
d d 
57. a b a f b 
e e 
ec c g d 
58. a b a e 
) 
h 
ra d f g 


59. The complementary graph G of a simple graph G has 
the same vertices as G. Two vertices are adjacent in G if 
and only if they are not adjacent in G. Describe each of 
these graphs. 

a) Kn b) Kinn ©) Ch d) Qn 


60. If G is a simple graph with 15 edges and G has 13 edges, 
how many vertices does G have? 
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61. 


62. 


63. 


* 64, 


65. 


* 66. 
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If the simple graph G has v vertices and e edges, how 
many edges does G have? 

If the degree sequence of the simple graph G is 
4,3, 3,2, 2, what is the degree sequence of G? 

If the degree sequence of the simple graph G is 
d,,do,..., dy, what is the degree sequence of G? 

Show that if G is a bipartite simple graph with v vertices 
and e edges, then e < v?/4. 

Show that if G is a simple graph with n vertices, then the 
union of G and G is Kn. 

Describe an algorithm to decide whether a graph is bipar- 
tite based on the fact that a graph is bipartite if and only 
if it is possible to color its vertices two different colors 
so that no two vertices of the same color are adjacent. 


The converse of a directed graph G = (V, E), denoted 
by G”", is the directed graph (V, F), where the set F 
of edges of G°°”” is obtained by reversing the direction of 
each edge in E. 


67. 


Draw the converse of each of the graphs in Exercises 7—9 
in Section 10.1. 


68. 


69. 


70. 


71. 


72. 


73. 


Show that (G°"")°°"” = G whenever G is a directed 
graph. 

Show that the graph G is its own converse if and only if the 
relation associated with G (see Section 9.3) is symmetric. 
Show that if a bipartite graph G = (V, E) is n-regular for 
some positive integer n (see the preamble to Exercise 53) 
and (V,, V2) is a bipartition of V, then |V;| = | V2|. That 
is, show that the two sets in a bipartition of the vertex set 
of an n-regular graph must contain the same number of 
vertices. 


Draw the mesh network for interconnecting nine parallel 
processors. 

Ina variant of a mesh network for interconnecting n = m 
processors, processor P (i, j) is connected to the four pro- 
cessors P((i + 1) mod m, j) and P(i, (j + 1) mod m), 
so that connections wrap around the edges of the mesh. 
Draw this variant of the mesh network for 16 processors. 


2 


Show that every pair of processors in a mesh network 
of n = m? processors can communicate using O(./n) = 
O(m) hops between directly connected processors. 


Representing Graphs and Graph Isomorphism 


Introduction 


There are many useful ways to represent graphs. As we will see throughout this chapter, in 
working with a graph it is helpful to be able to choose its most convenient representation. In 
this section we will show how to represent graphs in several different ways. 

Sometimes, two graphs have exactly the same form, in the sense that there is a one-to-one 
correspondence between their vertex sets that preserves edges. In such a case, we say that the 
two graphs are isomorphic. Determining whether two graphs are isomorphic is an important 
problem of graph theory that we will study in this section. 


Representing Graphs 


One way to represent a graph without multiple edges is to list all the edges of this graph. Another 
way to represent a graph with no multiple edges is to use adjacency lists, which specify the 
vertices that are adjacent to each vertex of the graph. 


EXAMPLE 1 


Use adjacency lists to describe the simple graph given in Figure 1. 


Solution: Table | lists those vertices adjacent to each of the vertices of the graph. < 


e d 


FIGURE 1 A Simple Graph. 


EXAMPLE 2 


a b 


€ d 


FIGURE 3 
Simple Graph. 


Links 


EXAMPLE 3 


EXAMPLE 4 
a b 


d c 


FIGURE 4 

A Graph with the 
Given Adjacency 
Matrix. 
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b TABLE 2 An Adjacency List for a 
Directed Graph. 


Initial Vertex Terminal Vertices 


Q 
is) 


a 

b 

iC 

e d d 
FIGURE 2. A Directed Graph. : 


Represent the directed graph shown in Figure 2 by listing all the vertices that are the terminal 
vertices of edges starting at each vertex of the graph. 


Solution: Table 2 represents the directed graph shown in Figure 2. < 
Adjacency Matrices 


Carrying out graph algorithms using the representation of graphs by lists of edges, or by adja- 
cency lists, can be cumbersome if there are many edges in the graph. To simplify computation, 
graphs can be represented using matrices. Two types of matrices commonly used to represent 
graphs will be presented here. One is based on the adjacency of vertices, and the other is based 
on incidence of vertices and edges. 

Suppose that G = (V, E) is a simple graph where |V| = n. Suppose that the vertices of 
G are listed arbitrarily as v1, v2,..., V,. The adjacency matrix A (or Ag) of G, with respect 
to this listing of the vertices, is the n x n zero—one matrix with | as its (i, j)th entry when v; 
and v; are adjacent, and 0 as its (i, j)th entry when they are not adjacent. In other words, if its 
adjacency matrix is A = [a;;], then 


ee 1 if {yj, vj} is an edge of G, 
"10 otherwise. 


Use an adjacency matrix to represent the graph shown in Figure 3. 


Solution: We order the vertices as a, b, c, d. The matrix representing this graph is 


See © 
oror 
oCoOoOrrF 
ocor 


Draw a graph with the adjacency matrix 


orr Oo 
eH oor 
- OCF. 
orr oO 


with respect to the ordering of vertices a, b,c, d. 


Solution: A graph with this adjacency matrix is shown in Figure 4. < 
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EXAMPLE 5 


a b 


d c 


FIGURE 5 
A Pseudograph. 


Note that an adjacency matrix of a graph is based on the ordering chosen for the vertices. 
Hence, there may be as many as n! different adjacency matrices for a graph with n vertices, 
because there are n! different orderings of n vertices. 

The adjacency matrix of a simple graph is symmetric, that is, aj; = aj;, because both of 
these entries are 1 when v; and v; are adjacent, and both are 0 otherwise. Furthermore, because 
a simple graph has no loops, each entry a;;,i = 1,2,3,...,n, is 0. 

Adjacency matrices can also be used to represent undirected graphs with loops and with 
multiple edges. A loop at the vertex v; is represented by a | at the (i, i)th position of the 
adjacency matrix. When multiple edges connecting the same pair of vertices v; and v;, or 
multiple loops at the same vertex, are present, the adjacency matrix is no longer a zero—one 
matrix, because the (i, j)th entry of this matrix equals the number of edges that are associated 
to {v;, vj}. All undirected graphs, including multigraphs and pseudographs, have symmetric 
adjacency matrices. 


Use an adjacency matrix to represent the pseudograph shown in Figure 5. 


Solution: The adjacency matrix using the ordering of vertices a, b, c, d is 


NOW © 
pr OW 
Ne O 
oOoNrFN 


< 


We used zero—one matrices in Chapter 9 to represent directed graphs. The matrix for a 
directed graph G = (V, E) has a 1| in its (i, j)th position if there is an edge from 1; to vj, 
where vj, V2,..., Vy iS an arbitrary listing of the vertices of the directed graph. In other words, 
if A = [aj;] is the adjacency matrix for the directed graph with respect to this listing of the 
vertices, then 


1 if (vj, vj) is an edge of G, 
a; = ; 
zi 0 otherwise. 


The adjacency matrix for a directed graph does not have to be symmetric, because there may 
not be an edge from v; to v; when there is an edge from vj; to v;. 

Adjacency matrices can also be used to represent directed multigraphs. Again, such matrices 
are not zero—one matrices when there are multiple edges in the same direction connecting two 
vertices. In the adjacency matrix for a directed multigraph, a;; equals the number of edges that 
are associated to (vj, vj). 


TRADE-OFFS BETWEEN ADJACENCY LISTS AND ADJACENCY MATRICES When 
a simple graph contains relatively few edges, that is, when it is sparse, it is usually preferable 
to use adjacency lists rather than an adjacency matrix to represent the graph. For example, if 
each vertex has degree not exceeding c, where c is a constant much smaller than n, then each 
adjacency list contains c or fewer vertices. Hence, there are no more than cn items in all these 
adjacency lists. On the other hand, the adjacency matrix for the graph has n” entries. Note, 
however, that the adjacency matrix of a sparse graph is a sparse matrix, that is, a matrix with 
few nonzero entries, and there are special techniques for representing, and computing with, 
sparse matrices. 

Now suppose that a simple graph is dense, that is, suppose that it contains many edges, such 
as a graph that contains more than half of all possible edges. In this case, using an adjacency 
matrix to represent the graph is usually preferable over using adjacency lists. To see why, we 
compare the complexity of determining whether the possible edge {v;, vj} is present. Using an 
adjacency matrix, we can determine whether this edge is present by examining the (i, j)th entry 


EXAMPLE 6 


0) & Vv. 
& 
ey ee 
eo 
V4 V5 
FIGURE6 An 


Undirected 
Graph. 


EXAMPLE 7 


FIGURE 7 
A Pseudograph. 
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in the matrix. This entry is | if the graph contains this edge and is 0 otherwise. Consequently, 
we need make only one comparison, namely, comparing this entry with 0, to determine whether 
this edge is present. On the other hand, when we use adjacency lists to represent the graph, we 
need to search the list of vertices adjacent to either v; or v; to determine whether this edge is 
present. This can require ©(|V|) comparisons when many edges are present. 


Incidence Matrices 
Another common way to represent graphs is to use incidence matrices. Let G = (V, E) be an 
undirected graph. Suppose that vj, v2,..., Vv, are the vertices and e, e2,..., @m are the edges 


of G. Then the incidence matrix with respect to this ordering of V and E is the n x m matrix 
M = [m;;], where 


1 when edge e; is incident with v;, 
pies : : 
4 0 otherwise. 


Represent the graph shown in Figure 6 with an incidence matrix. 


Solution: The incidence matrix is 


1 | i it OO OW 2G 

w/O0 01101 

4/0 00 01 1/4. 

ull 0 1.0 0 0 

v101 01 1 0 < 


Incidence matrices can also be used to represent multiple edges and loops. Multiple edges 
are represented in the incidence matrix using columns with identical entries, because these edges 
are incident with the same pair of vertices. Loops are represented using a column with exactly 
one entry equal to 1, corresponding to the vertex that is incident with this loop. 


Represent the pseudograph shown in Figure 7 using an incidence matrix. 


’s Solution: The incidence matrix for this graph is 


€} €2 €3 €4 €5 €& e7 


ooorrf 
ooorr 
oorr co 
= OF © 
pe OOrO 
orore 


Isomorphism of Graphs 


We often need to know whether it is possible to draw two graphs in the same way. That is, do 
the graphs have the same structure when we ignore the identities of their vertices? For instance, 
in chemistry, graphs are used to model chemical compounds (in a way we will describe later). 
Different compounds can have the same molecular formula but can differ in structure. Such 
compounds can be represented by graphs that cannot be drawn in the same way. The graphs 
representing previously known compounds can be used to determine whether a supposedly new 
compound has been studied before. 
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There is a useful terminology for graphs with the same structure. 


The simple graphs G; = (V,, E,) and Gz = (V2, E2) are isomorphic if there exists a one- 
to-one and onto function f from Vj to V2 with the property that a and b are adjacent in G, if 
and only if f(a) and f(b) are adjacent in G2, for all a and b in V;. Such a function f is called 
an isomorphism.* Two simple graphs that are not isomorphic are called nonisomorphic. 


In other words, when two simple graphs are isomorphic, there is a one-to-one correspondence 
between vertices of the two graphs that preserves the adjacency relationship. Isomorphism of 
simple graphs is an equivalence relation. (We leave the verification of this as Exercise 45.) 


Show that the graphs G = (V, E) and H = (W, F), displayed in Figure 8, are isomorphic. 


Solution: The function f with f(u,) = v1, f(u2) = va, f(u3) = v3, and f(u4) = v2 Is a one- 
to-one correspondence between V and W. To see that this correspondence preserves adjacency, 
note that adjacent vertices in G are uv; and uz, uv, and u3, uz and w4, and w3 and w4, and each of the 
pairs f(u,) = vy and f(u2) = va, f(u1) = v1 and f(u3) = v3, f(u2) = va and f(u4) = vo, 
and f(u3) = v3 and f (u4) = v2 consists of two adjacent vertices in H. < 


Determining whether Two Simple Graphs are Isomorphic 


It is often difficult to determine whether two simple graphs are isomorphic. There are n! possible 
one-to-one correspondences between the vertex sets of two simple graphs with n vertices. Testing 
each such correspondence to see whether it preserves adjacency and nonadjacency is impractical 
if n is at all large. 

Sometimes it is not hard to show that two graphs are not isomorphic. In particular, we can 
show that two graphs are not isomorphic if we can find a property only one of the two graphs 
has, but that is preserved by isomorphism. A property preserved by isomorphism of graphs is 
called a graph invariant. For instance, isomorphic simple graphs must have the same number 
of vertices, because there is a one-to-one correspondence between the sets of vertices of the 
graphs. 

Isomorphic simple graphs also must have the same number of edges, because the one-to-one 
correspondence between vertices establishes a one-to-one correspondence between edges. In 
addition, the degrees of the vertices in isomorphic simple graphs must be the same. That is, a 
vertex v of degree d in G must correspond to a vertex f(v) of degree d in H, because a vertex 
w in G is adjacent to v if and only if f(v) and f(w) are adjacent in H. 


Show that the graphs displayed in Figure 9 are not isomorphic. 


Solution: Both G and H have five vertices and six edges. However, H has a vertex of degree one, 
namely, e, whereas G has no vertices of degree one. It follows that G and H are not isomorphic. 


The number of vertices, the number of edges, and the number of vertices of each degree 
are all invariants under isomorphism. If any of these quantities differ in two simple graphs, 
these graphs cannot be isomorphic. However, when these invariants are the same, it does not 
necessarily mean that the two graphs are isomorphic. There are no useful sets of invariants 
currently known that can be used to determine whether simple graphs are isomorphic. 


*The word isomorphism comes from the Greek roots isos for “equal” and morphe for “form.” 
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gj Q 
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FIGURE9 The Graphs G and H. 
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H G H 


FIGURE 10 The Graphs G and H. 


Determine whether the graphs shown in Figure 10 are isomorphic. 


Solution: The graphs G and H both have eight vertices and 10 edges. They also both have four 
vertices of degree two and four of degree three. Because these invariants all agree, it is still 
conceivable that these graphs are isomorphic. 

However, G and H are not isomorphic. To see this, note that because deg(a) = 2 in G, a 
must correspond to either rt, u, x, or y in H, because these are the vertices of degree two in H. 
However, each of these four vertices in H is adjacent to another vertex of degree two in H, 
which is not true for a in G. 

Another way to see that G and H are not isomorphic is to note that the subgraphs of G 
and H made up of vertices of degree three and the edges connecting them must be isomorphic 
if these two graphs are isomorphic (the reader should verify this). However, these subgraphs, 
shown in Figure 11, are not isomorphic. 


To show that a function f from the vertex set of a graph G to the vertex set of a graph H/ is an 
isomorphism, we need to show that f preserves the presence and absence of edges. One helpful 
way to do this is to use adjacency matrices. In particular, to show that f is an isomorphism, we 
can show that the adjacency matrix of G is the same as the adjacency matrix of H, when rows 
and columns are labeled to correspond to the images under f of the vertices in G that are the 
labels of these rows and columns in the adjacency matrix of G. We illustrate how this is done 
in Example 11. 


Determine whether the graphs G and H displayed in Figure 12 are isomorphic. 


Solution: Both G and H have six vertices and seven edges. Both have four vertices of degree two 
and two vertices of degree three. It is also easy to see that the subgraphs of G and H consisting 
of all vertices of degree two and the edges connecting them are isomorphic (as the reader should 
verify). Because G and H agree with respect to these invariants, it is reasonable to try to find 
an isomorphism f/f. 


uy Uy vy V3 


Us 


U4 U3 V5 V4 


FIGURE 12. Graphs G and H. 
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We now will define a function f and then determine whether it is an isomorphism. Because 
deg(u;) = 2 and because iu is not adjacent to any other vertex of degree two, the image of u1 
must be either v4 or vg, the only vertices of degree two in H not adjacent to a vertex of degree 
two. We arbitrarily set f(u1) = ve. [If we found that this choice did not lead to isomorphism, 
we would then try f(u1) = v4.] Because wz is adjacent to u,, the possible images of w2 are v3 
and v5. We arbitrarily set f(u2) = v3. Continuing in this way, using adjacency of vertices and 
degrees as a guide, we set f(u3) = va, f (ua) = v5, f(us) = v1, and f (ue) = v2. We now have 
a one-to-one correspondence between the vertex set of G and the vertex set of H, namely, 
fui) = vo, f(u2) = v3, f(u3) = v4, f (ua) = V5, f(us) = 11, f (U6) = V2. To see whether f 
preserves edges, we examine the adjacency matrix of G, 


uy; ug U3 Ug U5 U6 


wu |O 1 0 1 0 0 
wll Oo 1 0 0 1 
Aga {2 1 9 1 0 0 
us| 1 0 1 0 1 0 |’ 
us} 0 0 0 1 0 | 
uw lO 1 0 0 1 0 


and the adjacency matrix of H with the rows and columns labeled by the images of the corre- 
sponding vertices in G, 


V6 V3 VW VWs Vy V2 


at 10 1 0 e| 
V3 1 0 1 0 0 1 
Aya {o 1 0 1 0 0f 

vs] 0 1 0 2 ail 
000 1 0 1 

lo c QO a | 


Because Ag = Aux, it follows that f preserves edges. We conclude that f is an isomorphism, 
so G and H are isomorphic. Note that if f turned out not to be an isomorphism, we would 
not have established that G and H are not isomorphic, because another correspondence of the 
vertices in G and H may be an isomorphism. 


ALGORITHMS FOR GRAPH ISOMORPHISM The best algorithms known for determining 
whether two graphs are isomorphic have exponential worst-case time complexity (in the number 
of vertices of the graphs). However, linear average-case time complexity algorithms are known 
that solve this problem, and there is some hope, but also skepticism, that an algorithm with 
polynomial worst-case time complexity for determining whether two graphs are isomorphic can 
be found. The best practical general purpose software for isomorphism testing, called NAUTY, 
can be used to determine whether two graphs with as many as 100 vertices are isomorphic 
in less than a second on a modern PC. NAUTY software can be downloaded over the Internet 
and experimented with. Practical algorithms for determining whether two graphs are isomorphic 
exist for graphs that are restricted in various ways, such as when the maximum degree of vertices 
is small. The problem of determining whether any two graphs are isomorphic is of special interest 
because it is one of only a few NP problems (see Exercise 72) not known to be either tractable 
or NP-complete (see Section 3.3). 


APPLICATIONS OF GRAPH ISOMORPHISMS_ Graph isomorphisms, and functions that 
are almost graph isomorphisms, arise in applications of graph theory to chemistry and to the 
design of electronic circuits, and other areas including bioinformatics and computer vision. 
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Chemists use multigraphs, known as molecular graphs, to model chemical compounds. In these 
graphs, vertices represent atoms and edges represent chemical bonds between these atoms. 
Two structural isomers, molecules with identical molecular formulas but with atoms bonded 
differently, have nonisomorphic molecular graphs. When a potentially new chemical compound 
is synthesized, a database of molecular graphs is checked to see whether the molecular graph 
of the compound is the same as one already known. 

Electronic circuits are modeled using graphs in which vertices represent components and 
edges represent connections between them. Modern integrated circuits, known as chips, are 
miniaturized electronic circuits, often with millions of transistors and connections between 
them. Because of the complexity of modern chips, automation tools are used to design them. 
Graph isomorphism is the basis for the verification that a particular layout of a circuit produced 
by an automated tool corresponds to the original schematic of the design. Graph isomorphism 
can also be used to determine whether a chip from one vendor includes intellectual property 
from a different vendor. This can be done by looking for large isomorphic subgraphs in the 
graphs modeling these chips. 


Exercises 
In Exercises 1-4 use an adjacency list to represent the given 12/1; 1 1 °0 
graph. 0 0 1 0 
1 0 1 0 

la b 2a b c 1 1 1 0 
In Exercises 13-15 represent the given graph using an adja- 
cency matrix. 

c d d e 13. a b 14, 4 b 
3. 


trix. 

9. Represent each of these graphs with an adjacency matrix. 16.) 3 2 17.1 201 
a) K4 b) Ki4 c) K2,3 3 0 4 2 0 3 0 
d) C4 e) W4 f) Q3 2 4 0 03 1 1 

; ; 1 0 1 =O 

In Exercises 10-12 draw a graph with the given adjacency 

matrix. 18 

“JO 1 3 0 4 

10.fo 1 «0 W/o 0 1 1 12 13 0 
1 0 1 0 0 1 =O 3 1 1 0 1 

0 1 +0 1 1 0 1 0 3 0 0 2 

1 1 1 0 4 01 2 3 


15. 


. Represent the graph in Exercise 1 with an adjacency ma- 


trix. 


. Represent the graph in Exercise 2 with an adjacency ma- 


trix. 


. Represent the graph in Exercise 3 with an adjacency ma- 


trix. 


. Represent the graph in Exercise 4 with an adjacency ma- 


In Exercises 16-18 draw an undirected graph represented by 
the given adjacency matrix. 
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In Exercises 19-21 find the adjacency matrix of the given 
directed multigraph with respect to the vertices listed in al- 
phabetic order. 


19. 


a 


21. a b 


c d 


In Exercises 22—24 draw the graph represented by the given 
adjacency matrix. 


22. 24. 


101 23./1 2 1 02 3 ~0 
001 2 0 0 1221 
111 0 2 2 2 1 1 =~0 

100 2 


25. Is every zero—one square matrix that is symmetric and 
has zeros on the diagonal the adjacency matrix of a sim- 
ple graph? 

26. Use an incidence matrix to represent the graphs in Exer- 
cises | and 2. 


27. Use an incidence matrix to represent the graphs in Exer- 
cises 13-15. 


* 28. What is the sum of the entries in a row of the adjacency 
matrix for an undirected graph? For a directed graph? 


* 29. What is the sum of the entries in a column of the adjacency 
matrix for an undirected graph? For a directed graph? 


30. What is the sum of the entries in a row of the incidence 
matrix for an undirected graph? 


31. What is the sum of the entries in a column of the incidence 
matrix for an undirected graph? 
* 32. Find an adjacency matrix for each of these graphs. 
a) Ky b) Ch c) Wr d) Kin.n e) On 
* 33. Find incidence matrices for the graphs in parts (a)—(d) of 
Exercise 32. 


In Exercises 34—44 determine whether the given pair of graphs 
is isomorphic. Exhibit an isomorphism or provide a rigorous 
argument that none exists. 


34. 


35. 


36. 


38. 


39. 


40. 


Vo 
Uy 

Vy v3 

uy U3 
, ; 
al 
Dx] SS | 
j 


uy 1) 


= 
S 


Us 


~ 
is) 


al 


Vo 
V5 
V3 


> "3 . 


41. 


42. 


43. 


44, 


45. 


46. 


47. 


48. 


49. 


uy Uy U3 Us U6 Ug 
e © | o ra) 
Ug uz 
vy Vo V4 Vig Vo Vg 
e | o © © 
V3 V7 
U6 uz 
uy Uy U3 U4 Us 
° © 
Ug Ug U9 
V6 V7 


Vg Vo Vio 


Ug 


uz U3 V7 V3 


U6 U4 V% V4 


Show that isomorphism of simple graphs is an equiva- 
lence relation. 


Suppose that G and H are isomorphic simple graphs. 
Show that their complementary graphs G and H are also 
isomorphic. 


Describe the row and column of an adjacency matrix of 
a graph corresponding to an isolated vertex. 


Describe the row of an incidence matrix of a graph cor- 
responding to an isolated vertex. 


Show that the vertices of a bipartite graph with two or 
more vertices can be ordered so that its adjacency matrix 
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has the form 


[s 0] 


where the four entries shown are rectangular blocks. 
A simple graph G is called self-complementary if G and G 
are isomorphic. 


50. Show that this graph is self-complementary. 


a b 
d c 

51. Find a self-complementary simple graph with five ver- 
tices. 


*52. Show that if G is a self-complementary simple graph 

with v vertices, then v = 0 or | (mod 4). 

53. For which integers n is C,, self-complementary? 

54. How many nonisomorphic simple graphs are there with n 
vertices, when n is 
a) 2? b) 3? c) 4? 

55. How many nonisomorphic simple graphs are there with 
five vertices and three edges? 

56. How many nonisomorphic simple graphs are there with 
six vertices and four edges? 

57. Are the simple graphs with the following adjacency ma- 
trices isomorphic? 


a) 10 0 1 Oo 1 1 
0 0 1].}1 0 0 
1 1 0 0 0 

b)/0 1 0 1 0 11 1 
100 1) |1 00 1 
0 0 0 1f’J1 00 1 
1 1 1 O} |1 1 1 =«0 

c/o 1 1 0 0 10 1 
1 0 0 1] |1 0 0 0 
1 0 0 1]’°]0 00 1 
0 1 1 OO} |1 0 1 0 


58. Determine whether the graphs without loops with these 
incidence matrices are isomorphic. 


a) /1 0 1] }1 1 +0 


Oo 1 1y,J1 0 1 
1 1 0 0 1 1 

b)/1 1 0 0 0 0 1 0 0 1 
1 0 1 0 1 0 1 1 1 +0 
0 0 0 1 1]’}1 0 0 1 °0 
0 1 1 1 =+0 1 0 1 0 1 


59, Extend the definition of isomorphism of simple graphs to 
undirected graphs containing loops and multiple edges. 


60. Define isomorphism of directed graphs. 
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In Exercises 61-64 determine whether the given pair of di- 
rected graphs are isomorphic. (See Exercise 60.) 


61. uy Uy Vy Vo 
66. 
67. 
U3 U4 V3 V4 
62. 1 ug vy Vo 
x | 7 
U3 U4 V3 V4 
* 
63. i ‘A 69. 
| | . | | . *70. 
Uy U3 Vo V3 


64. 


POX 


1 ug U3 

U4 Us U6 
vy Vo 

, <> ; 
V5 V4 


Connectivity 


65. 


71. 


72. 


Show that if G and H are isomorphic directed graphs, 
then the converses of G and H (defined in the preamble 
of Exercise 67 of Section 10.2) are also isomorphic. 


Show that the property that a graph is bipartite is an iso- 
morphic invariant. 


Find a pair of nonisomorphic graphs with the same de- 
gree sequence (defined in the preamble to Exercise 36 
in Section 10.2) such that one graph is bipartite, but the 
other graph is not bipartite. 


How many nonisomorphic directed simple graphs are 
there with n vertices, when n is 


a) 2? b) 3? c) 4? 


What is the product of the incidence matrix and its trans- 
pose for an undirected graph? 


How much storage is needed to represent a simple graph 
with n vertices and m edges using 


a) adjacency lists? 
b) an adjacency matrix? 


c) an incidence matrix? 


A devil’s pair for a purported isomorphism test is a pair of 
nonisomorphic graphs that the test fails to show that they are 
not isomorphic. 


Find a devil’s pair for the test that checks the degree se- 
quence (defined in the preamble to Exercise 36 in Sec- 
tion 10.2) in two graphs to make sure they agree. 


Suppose that the function f from V; to V2 is an isomor- 
phism of the graphs G; = (V,, E,) and G2 = (V2, E2). 
Show that it is possible to verify this fact in time polyno- 
mial in terms of the number of vertices of the graph, in 
terms of the number of comparisons needed. 


Introduction 


Many problems can be modeled with paths formed by traveling along the edges of graphs. For 
instance, the problem of determining whether a message can be sent between two computers 
using intermediate links can be studied with a graph model. Problems of efficiently planning 
routes for mail delivery, garbage pickup, diagnostics in computer networks, and so on can be 
solved using models that involve paths in graphs. 


Paths 


Informally, a path is a sequence of edges that begins at a vertex of a graph and travels from 
vertex to vertex along edges of the graph. As the path travels along its edges, it visits the vertices 
along this path, that is, the endpoints of these edges. 


DEFINITION 1 


EXAMPLE 1 


10.4 Connectivity 679 


A formal definition of paths and related terminology is given in Definition 1. 


Let n be a nonnegative integer and G an undirected graph. A path of length n from u 
to v in G is a sequence of n edges e1,...,e, of G for which there exists a sequence 
XQ =U,X1,.--,Xn—1, Xn = Vv Of vertices such that e; has, fori = 1,...,n, the endpoints x;_ 
and x;. When the graph is simple, we denote this path by its vertex sequence x9, X1,..., Xn 
(because listing these vertices uniquely determines the path). The path is a circuit if it begins 
and ends at the same vertex, that is, if = v, and has length greater than zero. The path or cir- 
cuit is said to pass through the vertices x1, x2, ..., X,—1 Or traverse the edges e1, €2,..., €n.- 
A path or circuit is simple if it does not contain the same edge more than once. 


When it is not necessary to distinguish between multiple edges, we will denote a path 
€1,€2,.--,n, where e; is associated with {x;_1, xj} fori? = 1,2,...,n by its vertex sequence 
X0,X1,...,Xn. This notation identifies a path only as far as which vertices it passes through. 
Consequently, it does not specify a unique path when there is more than one path that passes 
through this sequence of vertices, which will happen if and only if there are multiple edges 
between some successive vertices in the list. Note that a path of length zero consists of a single 
vertex. 


Remark: There is considerable variation of terminology concerning the concepts defined 
in Definition 1. For instance, in some books, the term walk is used instead of path, 
where a walk is defined to be an alternating sequence of vertices and edges of a graph, 
VO; 1, V1; €2,--+; Vn—1; ns Yn, Where vj_; and v; are the endpoints of e; fori = 1,2,...,n. 
When this terminology is used, closed walk is used instead of circuit to indicate a walk that 
begins and ends at the same vertex, and trail is used to denote a walk that has no repeated 
edge (replacing the term simple path). When this terminology is used, the terminology path is 
often used for a trail with no repeated vertices, conflicting with the terminology in Definition 1. 
Because of this variation in terminology, you will need to make sure which set of definitions are 
used in a particular book or article when you read about traversing edges of a graph. The text 
[GrYe06] is a good reference for the alternative terminology described in this remark. 


In the simple graph shown in Figure |, a, d, c, f, e is a simple path of length 4, because {a, d}, 
{d, c}, {c, f}, and {f, e} are all edges. However, d, e, c, a is not a path, because {e, c} is not an 
edge. Note that b, c, f, e, b is a circuit of length 4 because {b, c}, {c, f}, {f, e}, and {e, b} are 
edges, and this path begins and ends at b. The path a, b, e, d, a, b, which is of length 5, is not 
simple because it contains the edge {a, b} twice. < 


Paths and circuits in directed graphs were introduced in Chapter 9. We now provide more 
general definitions. 


FIGURE 1 A Simple Graph. 
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DEFINITION 2 


EXAMPLE 2 


Links 


EXAMPLE 3 


Links 


Replace Kevin Bacon by 
your own favorite actor to 
invent a new party game 


Let n be a nonnegative integer and G a directed graph. A path of length n from u to vin G isa 
sequence of edges e), e2, ..., €n of G such that e; 1s associated with (xo, x1), e2 1s associated 
with (x1, x2), and so on, with e, associated with (x)_1, Xn), where x9 = u and x, = v. When 
there are no multiple edges in the directed graph, this path is denoted by its vertex sequence 
Xo, X1,X2,..-,Xn,.A path of length greater than zero that begins and ends at the same vertex 
is called a circuit or cycle. A path or circuit is called simple if it does not contain the same 
edge more than once. 


Remark: Terminology other than that given in Definition 2 is often used for the concepts defined 
there. In particular, the alternative terminology that uses walk, closed walk, trail, and path 
(described in the remarks following Definition 1) may be used for directed graphs. See [GrYe05] 
for details. 


Note that the terminal vertex of an edge in a path is the initial vertex of the next edge in the 
path. When it is not necessary to distinguish between multiple edges, we will denote a path 
€1, €2,.--, €n, Where e; is associated with (x;-1, xj) fori = 1,2,...,n, by its vertex sequence 
X0,X],--.,X,. The notation identifies a path only as far as which the vertices it passes through. 
There may be more than one path that passes through this sequence of vertices, which will 
happen if and only if there are multiple edges between two successive vertices in the list. 
Paths represent useful information in many graph models, as Examples 2—4 demonstrate. 


Paths in Acquaintanceship Graphs In an acquaintanceship graph there is a path between 
two people if there is a chain of people linking these people, where two people adjacent in 
the chain know one another. For example, in Figure 6 in Section 10.1, there is a chain of six 
people linking Kamini and Ching. Many social scientists have conjectured that almost every 
pair of people in the world are linked by a small chain of people, perhaps containing just five or 
fewer people. This would mean that almost every pair of vertices in the acquaintanceship graph 
containing all people in the world is linked by a path of length not exceeding four. The play Six 
Degrees of Separation by John Guare is based on this notion. < 


Paths in Collaboration Graphs In a collaboration graph, two people a and b are connected 
by a path when there is a sequence of people starting with a and ending with b such that the 
endpoints of each edge in the path are people who have collaborated. We will consider two 
particular collaboration graphs here. First, in the academic collaboration graph of people who 
have written papers in mathematics, the Erdés number of a person m (defined in terms of 
relations in Supplementary Exercise 14 in Chapter 9) is the length of the shortest path between 
m and the extremely prolific mathematician Paul Erd6s (who died in 1996). That is, the Erdés 
number of a mathematician is the length of the shortest chain of mathematicians that begins 
with Paul Erdés and ends with this mathematician, where each adjacent pair of mathematicians 
have written a joint paper. The number of mathematicians with each Erd6és number as of early 
2006, according to the Erdés Number Project, is shown in Table 1. 

In the Hollywood graph (see Example 3 in Section 10.1) two actors a and b are linked when 
there is a chain of actors linking a and b, where every two actors adjacent in the chain have 
acted in the same movie. In the Hollywood graph, the Bacon number of an actor c is defined 
to be the length of the shortest path connecting c and the well-known actor Kevin Bacon. As 
new movies are made, including new ones with Kevin Bacon, the Bacon number of actors can 
change. In Table 2 we show the number of actors with each Bacon number as of early 2011 
using data from the Oracle of Bacon website. The origins of the Bacon number of an actor 
dates back to the early 1990s, when Kevin Bacon remarked that he had worked with everyone 
in Hollywood or someone who worked with them. This lead some people to invent a party 
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TABLE 1 The Number TABLE 2 The Number 
of Mathematicians of Actors with a Given 
with a Given Erdés Bacon Number (as of 
Number (as of early early 2011). 


2006). 
Bacon Number 
Number Number of People 


Number of People 
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2,367 
242,407 
785,389 
200,602 
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1,277 
114 
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game where participants where challenged to find a sequence of movies leading from each actor 
named to Kevin Bacon. We can find a number similar to a Bacon number using any actor as the 
center of the acting universe. < 


Connectedness in Undirected Graphs 


When does a computer network have the property that every pair of computers can share infor- 
mation, if messages can be sent through one or more intermediate computers? When a graph 
is used to represent this computer network, where vertices represent the computers and edges 
represent the communication links, this question becomes: When is there always a path between 
two vertices in the graph? 


An undirected graph is called connected if there is a path between every pair of distinct 
vertices of the graph. An undirected graph that is not connected is called disconnected. We 
say that we disconnect a graph when we remove vertices or edges, or both, to produce a 
disconnected subgraph. 


Thus, any two computers in the network can communicate if and only if the graph of this network 
is connected. 


The graph G; in Figure 2 is connected, because for every pair of distinct vertices there is a 
path between them (the reader should verify this). However, the graph G2 in Figure 2 is not 
connected. For instance, there is no path in G2 between vertices a and d. < 


We will need the following theorem in Chapter 11. 
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FIGURE 2 The Graphs G, and 
G2. 


FIGURE3_ The Graph 4 and Its 
Connected Components Hj, H2, and H3. 


There is a simple path between every pair of distinct vertices of a connected undirected graph. 


Proof: Let u and v be two distinct vertices of the connected undirected graph G = (V, E). Be- 
cause G is connected, there is at least one path between u and v. Let x9, x1, ..., Xn, where xp = u 
and x, = v, be the vertex sequence of a path of least length. This path of least length is simple. To 
see this, suppose it is not simple. Then x; = x; for some i and j withO <i < j. This means that 
there is a path from u to v of shorter length with vertex sequence xo, X1,...,Xj-1,Xj,.-+5%Xn 
obtained by deleting the edges corresponding to the vertex sequence x;,..., Xj-1. 


CONNECTED COMPONENTS A connected component of a graph G is a connected sub- 
graph of G that is not a proper subgraph of another connected subgraph of G. That is, aconnected 
component of a graph G is a maximal connected subgraph of G. A graph G that is not connected 
has two or more connected components that are disjoint and have G as their union. 


What are the connected components of the graph H shown in Figure 3? 


Solution: The graph 4 is the union of three disjoint connected subgraphs H), H2, and H3, shown 
in Figure 3. These three subgraphs are the connected components of H. < 


Connected Components of CallGraphs Two vertices x and y are in the same component of a 
telephone call graph (see Example 4 in Section 10.1) when there is a sequence of telephone calls 
beginning at x and ending at y. When a call graph for telephone calls made during a particular 
day in the AT&T network was analyzed, this graph was found to have 53,767,087 vertices, 
more than 170 million edges, and more than 3.7 million connected components. Most of these 
components were small; approximately three-fourths consisted of two vertices representing pairs 
of telephone numbers that called only each other. This graph has one huge connected component 
with 44,989,297 vertices comprising more than 80% of the total. Furthermore, every vertex in 
this component can be linked to any other vertex by a chain of no more than 20 calls. < 


How Connected is a Graph? 


Suppose that a graph represents a computer network. Knowing that this graph is connected tells 
us that any two computers on the network can communicate. However, we would also like to 
understand how reliable this network is. For instance, will it still be possible for all computers to 
communicate after a router or a communications link fails? To answer this and similar questions, 
we now develop some new concepts. 
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Sometimes the removal from a graph of a vertex and all incident edges produces a subgraph 
with more connected components. Such vertices are called cut vertices (or articulation points). 
The removal of a cut vertex from a connected graph produces a subgraph that is not connected. 
Analogously, an edge whose removal produces a graph with more connected components than 
in the original graph is called a cut edge or bridge. Note that in a graph representing a computer 
network, a cut vertex and a cut edge represent an essential router and an essential link that cannot 
fail for all computers to be able to communicate. 


Find the cut vertices and cut edges in the graph G; shown in Figure 4. 


Solution: The cut vertices of G; are b, c, and e. The removal of one of these vertices (and its 
adjacent edges) disconnects the graph. The cut edges are {a, b} and {c, e}. Removing either one 
of these edges disconnects G1. < 


VERTEX CONNECTIVITY Not all graphs have cut vertices. For example, the complete 
graph K,, where n > 3, has no cut vertices. When you remove a vertex from K,, and all edges 
incident to it, the resulting subgraph is the complete graph K,—1, a connected graph. Connected 
graphs without cut vertices are called nonseparable graphs, and can be thought of as more 
connected than those with a cut vertex. We can extend this notion by defining a more granulated 
measure of graph connectivity based on the minimum number of vertices that can be removed 
to disconnect a graph. 
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FIGURE 4 Some Connected Graphs 
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x is the lowercase Greek 
letter kappa. 
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A is the lowercase Greek 
letter lambda. 


EXAMPLE 9 


A subset V’ of the vertex set V of G = (V, E) is a vertex cut, or separating set, if G — V’ 
is disconnected. For instance, in the graph in Figure 1, the set {b, c, e} is a vertex cut with three 
vertices, as the reader should verify. We leave it to the reader (Exercise 51) to show that every 
connected graph, except a complete graph, has a vertex cut. We define the vertex connectivity 
of a noncomplete graph G, denoted by «(G), as the minimum number of vertices in a vertex 
cut. 

When G is acomplete graph, it has no vertex cuts, because removing any subset of its vertices 
and all incident edges still leaves a complete graph. Consequently, we cannot define « (G) as the 
minimum number of vertices in a vertex cut when G is complete. Instead, we set (K,) =n — 1, 
the number of vertices needed to be removed to produce a graph with a single vertex. 

Consequently, for every graph G, «(G) is minimum number of vertices that can be re- 
moved from G to either disconnect G or produce a graph with a single vertex. We have 
0<«(G) <n-—1if Ghasn vertices, «(G) = 0 if and only if G is disconnected or G = K1, 
and «(G) = n — 1 if and only if G is complete [see Exercise 52(a)]. 

The larger «(G) is, the more connected we consider G to be. Disconnected graphs and Ky 
have «(G) = 0, connected graphs with cut vertices and K2 have «(G) = 1, graphs without cut 
vertices that can be disconnected by removing two vertices and K3 have «(G) = 2, and so 
on. We say that a graph is k-connected (or k-vertex-connected), if «(G) > k. A graph G is 1- 
connected if it is connected and not a graph containing a single vertex; a graph is 2-connected, or 
biconnected, if it is nonseparable and has at least three vertices. Note that if G is a k-connected 
graph, then G is a j-connected graph for all j withO < j <k. 


Find the vertex connectivity for each of the graphs in Figure 4. 


Solution: Each of the five graphs in Figure 4 is connected and has more than vertex, so each 
of these graphs has positive vertex connectivity. Because G; is a connected graph with a cut 
vertex, as shown in Example 7, we know that «(G,) = 1. Similarly, «(G2) = 1, because c is a 
cut vertex of Go. 

The reader should verify that G3 has no cut vertices. but that {b, g} is a vertex cut. Hence, 
k(G3) = 2. Similarly, because G4 has a vertex cut of size two, {c, f}, but no cut vertices. It 
follows that « (G4) = 2. The reader can verify that Gs has no vertex cut of size two, but {b, c, f} 
is a vertex cut of G5. Hence, «(G5) = 3. 


EDGE CONNECTIVITY We can also measure the connectivity of a connected graph G = 
(V, E) in terms of the minimum number of edges that we can remove to disconnect it. If a graph 
has a cut edge, then we need only remove it to disconnect G. If G does not have a cut edge, 
we look for the smallest set of edges that can be removed to disconnect it. A set of edges E’ 
is called an edge cut of G if the subgraph G — E’ is disconnected. The edge connectivity 
of a graph G, denoted by A(G), is the minimum number of edges in an edge cut of G. This 
defines 4(G) for all connected graphs with more than one vertex because it is always possible 
to disconnect such a graph by removing all edges incident to one of its vertices. Note that 
A(G) = 0 if G is not connected. We also specify that 4(G) = 0 if G is a graph consisting of a 
single vertex. It follows that if G is a graph with n vertices, then 0 < A(G) <n — 1. We leave 
it to the reader [Exercise 52(b)] to show that A(G) = n — | where G is a graph with n vertices 
if and only if G = Ky, which is equivalent to the statement that 4(G) < n — 2 when G is nota 
complete graph. 


Find the edge connectivity of each of the graphs in Figure 4. 


Solution: Each of the five graphs in Figure 4 is connected and has more than one vertex, so we 
know that all of them have positive edge connectivity. As we saw in Example 7, G, has a cut 
edge, so A(G,) = 1. 
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The graph G2 has no cut edges, as the reader should verify, but the removal of the two edges 
{a, b} and {a, c} disconnects it. Hence, 4(G2) = 2. Similarly, 4(G3) = 2, because G3 has no 
cut edges, but the removal of the two edges {b, c} and {f, g} disconnects it. 

The reader should verify that the removal of no two edges disconnects G4, but the removal 
of the three edges {b, c}, {a, f}, and { f, g} disconnects it. Hence, (G4) = 3. Finally, the reader 
should verify that 4(G5) = 3, because the removal of any two of its edges does not disconnect 
it, but the removal of {a, b}, {a, g}, and {a, h} does. < 


AN INEQUALITY FOR VERTEX CONNECTIVITY AND EDGE CONNECTIVITY 
When G = (V, E) is anoncomplete connected graph with at least three vertices, the minimum 
degree of a vertex of G is an upper bound for both the vertex connectivity of G and the edge 
connectivity of G. That is, «(G) < minycy deg(v) and A(G) < min,cy deg(v). To see this, 
observe that deleting all the neighbors of a fixed vertex of minimum degree disconnects G, and 
deleting all the edges that have a fixed vertex of minimum degree as an endpoint disconnects G. 

In Exercise 55, we ask the reader to show that k(G) < 4(G) when G is a connected non- 
complete graph. Note also that k(K,) = A(Ky,) = minyey deg(v) = n — | when n is a positive 
integer and that «(G) = A(G) = 0 when G is a disconnected graph. Putting these facts together, 
establishes that for all graphs G, 


k(G) < A(G) < minyey deg(v). 


APPLICATIONS OF VERTEX AND EDGE CONNECTIVITY Graph connectivity plays 
an important role in many problems involving the reliability of networks. For instance, as we 
mentioned in our introduction of cut vertices and cut edges, we can model a data network using 
vertices to represent routers and edges to represent links between them. The vertex connectivity 
of the resulting graph equals the minimum number of routers that disconnect the network when 
they are out of service. If fewer routers are down, data transmission between every pair of routers 
is still possible. The edge connectivity represents the minimum number of fiber optic links that 
can be down to disconnect the network. If fewer links are down, it will still be possible for data 
to be transmitted between every pair of routers. 

We can model a highway network, using vertices to represent highway intersections and 
edges to represent sections of roads running between intersections. The vertex connectivity of the 
resulting graph represents the minimum number of intersections that can be closed at a particular 
time that makes it impossible to travel between every two intersections. If fewer intersections 
are closed, travel between every pair of intersections is still possible. The edge connectivity 
represents the minimum number of roads that can be closed to disconnect the highway network. 
If fewer highways are closed, it will still be possible to travel between any two intersections. 
Clearly, it would be useful for the highway department to take this information into account 
when planning road repairs. 


Connectedness in Directed Graphs 


There are two notions of connectedness in directed graphs, depending on whether the directions 
of the edges are considered. 


A directed graph is strongly connected if there is a path from a to b and from b to a whenever 
a and b are vertices in the graph. 


For a directed graph to be strongly connected there must be a sequence of directed edges 
from any vertex in the graph to any other vertex. A directed graph can fail to be strongly 
connected but still be in “one piece.” Definition 5 makes this notion precise. 


686 10/ Graphs 


DEFINITION 5 


EXAMPLE 10 


EXAMPLE 11 


EXAMPLE 12 


Links 


In 2010 the Web graph 
was estimated to have at 
least 55 billion vertices 
and one trillion edges. 
This implies that more 
than 40 TB of computer 
memory would have been 
needed to represent its 
adjacency matrix. 


A directed graph is weakly connected if there is a path between every two vertices in the 
underlying undirected graph. 


That is, a directed graph is weakly connected if and only if there is always a path between 
two vertices when the directions of the edges are disregarded. Clearly, any strongly connected 
directed graph is also weakly connected. 


Are the directed graphs G and H shown in Figure 5 strongly connected? Are they weakly 
connected? 


Solution: G is strongly connected because there is a path between any two vertices in this 
directed graph (the reader should verify this). Hence, G is also weakly connected. The graph 
H is not strongly connected. There is no directed path from a to b in this graph. However, H is 
weakly connected, because there is a path between any two vertices in the underlying undirected 
graph of H (the reader should verify this). 


STRONG COMPONENTS OFA DIRECTED GRAPH The subgraphs of a directed graph G 
that are strongly connected but not contained in larger strongly connected subgraphs, that is, 
the maximal strongly connected subgraphs, are called the strongly connected components or 
strong components of G. Note that if a and b are two vertices in a directed graph, their strong 
components are either the same or disjoint. (We leave the proof of this last fact as Exercise 17.) 


The graph H in Figure 5 has three strongly connected components, consisting of the vertex a; 
the vertex e; and the subgraph consisting of the vertices b, c, and d and edges (b, c), (c, d), and 
(d, b). < 


The Strongly Connected Components of the Web Graph The Web graph introduced in 
Example 5 of Section 10.1 represents Web pages with vertices and links with directed edges. A 
snapshot of the Web in 1999 produced a Web graph with over 200 million vertices and over 1.5 
billion edges (numbers that have now grown considerably). (See [Br00] for details.) 

The underlying undirected graph of this Web graph is not connected, but it has a connected 
component that includes approximately 90% of the vertices in the graph. The subgraph of the 
original directed graph corresponding to this connected component of the underlying undirected 
graph (that is, with the same vertices and all directed edges connecting vertices in this graph) 
has one very large strongly connected component and many small ones. The former is called 
the giant strongly connected component (GSCC) of the directed graph. A Web page in this 
component can be reached following links starting at any other page in this component. The 
GSCC in the Web graph produced by this study was found to have over 53 million vertices. 
The remaining vertices in the large connected component of the undirected graph represent 
three different types of Web pages: pages that can be reached from a page in the GSCC, but 
do not link back to these pages following a series of links; pages that link back to pages in the 
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FIGURE 5 The Directed Graphs G and H. 
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GSCC following a series of links, but cannot be reached by following links on pages in the 
GSCC; and pages that cannot reach pages in the GSCC and cannot be reached from pages in 
the GSCC following a series of links. In this study, each of these three other sets was found to 
have approximately 44 million vertices. (It is rather surprising that these three sets are close to 
the same size.) 


Paths and Isomorphism 


There are several ways that paths and circuits can help determine whether two graphs are 
isomorphic. For example, the existence of a simple circuit of a particular length is a useful 
invariant that can be used to show that two graphs are not isomorphic. In addition, paths can be 
used to construct mappings that may be isomorphisms. 

As we mentioned, a useful isomorphic invariant for simple graphs is the existence of a 
simple circuit of length k, where k is a positive integer greater than 2. (The proof that this is an 
invariant is left as Exercise 60.) Example 13 illustrates how this invariant can be used to show 
that two graphs are not isomorphic. 


Determine whether the graphs G and H shown in Figure 6 are isomorphic. 


Solution: Both G and H have six vertices and eight edges. Each has four vertices of degree 
three, and two vertices of degree two. So, the three invariants—number of vertices, number of 
edges, and degrees of vertices—all agree for the two graphs. However, H has a simple circuit 
of length three, namely, v1, v2, v6, v1, whereas G has no simple circuit of length three, as 
can be determined by inspection (all simple circuits in G have length at least four). Because 
the existence of a simple circuit of length three is an isomorphic invariant, G and H are not 
isomorphic. < 


We have shown how the existence of a type of path, namely, a simple circuit of a particular 
length, can be used to show that two graphs are not isomorphic. We can also use paths to find 
mappings that are potential isomorphisms. 


Determine whether the graphs G and H shown in Figure 7 are isomorphic. 
Solution: Both G and H have five vertices and six edges, both have two vertices of degree three 
and three vertices of degree two, and both have a simple circuit of length three, a simple circuit 


of length four, and a simple circuit of length five. Because all these isomorphic invariants agree, 
G and H may be isomorphic. 
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FIGURE 6 The Graphs G and H. FIGURE 7 The Graphs G and H. 
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FIGURE 8 The 
Graph G. 


To find a possible isomorphism, we can follow paths that go through all vertices so that the 
corresponding vertices in the two graphs have the same degree. For example, the paths w1, ua, 
U3, U2, Us in G and v3, v2, vj, V5, v4 in H both go through every vertex in the graph; start at a 
vertex of degree three; go through vertices of degrees two, three, and two, respectively; and end 
at a vertex of degree two. By following these paths through the graphs, we define the mapping f 
with f(u,) = v3, f(a) = v2, f(u3) = v1, f(u2) = vs, and f (us) = vq. The reader can show 
that f is an isomorphism, so G and H are isomorphic, either by showing that f preserves edges 
or by showing that with the appropriate orderings of vertices the adjacency matrices of G and H 
are the same. < 


Counting Paths Between Vertices 


The number of paths between two vertices in a graph can be determined using its adjacency 
matrix. 


Let G be a graph with adjacency matrix A with respect to the ordering vj, v2,..., Vp, of 
the vertices of the graph (with directed or undirected edges, with multiple edges and loops 
allowed). The number of different paths of length r from v; to v;, where r is a positive integer, 
equals the (7, 7)th entry of A’. 


Proof: The theorem will be proved using mathematical induction. Let G be a graph with adja- 
cency matrix A (assuming an ordering v1, v2, ..., V, of the vertices of G). The number of paths 
from v; to v; of length 1 is the (7, j)th entry of A, because this entry is the number of edges 
from vj to v;. 

Assume that the (i, j)th entry of A” is the number of different paths of length r from v; 
to v;. This is the inductive hypothesis. Because A’t! = A’A, the (i, j)th entry of A’t! equals 


bjaij + bj2d2j; + +++ + dindnj, 


where b;, is the (i, k)th entry of A’. By the inductive hypothesis, b;, is the number of paths of 
length r from vj to vz. 

A path of length r+ 1 from v; to vj is made up of a path of length r from v; to some 
intermediate vertex v,, and an edge from v, to v;. By the product rule for counting, the number 
of such paths is the product of the number of paths of length r from v; to vz, namely, bjz, and 
the number of edges from v,; to vj, namely, ag;. When these products are added for all possible 
intermediate vertices v;, the desired result follows by the sum rule for counting. 4 


How many paths of length four are there from a to d in the simple graph G in Figure 8? 


Solution: The adjacency matrix of G (ordering the vertices as a, b, c, d) is 
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Hence, the number of paths of length four from a to d is the (1, 4)th entry of A+. Because 
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Extra =~ there are exactly eight paths of length four from a to d. By inspection of the graph, we see that 


a,b,a,b,d;a,b,a,c,d;a,b,d,b,d;a,b,d,c,d;a,c,a,b,d;a,c,a,c,d;a,c,d,b,d; and 


a,c,d,c,d are the eight paths of length four from a to d. < 


Theorem 2 can be used to find the length of the shortest path between two vertices of a 
graph (see Exercise 56), and it can also be used to determine whether a graph is connected (see 


Exercises 61 and 62). 


Exercises 


1. Does each of these lists of vertices form a path in the 
following graph? Which paths are simple? Which are cir- 
cuits? What are the lengths of those that are paths? 


a) a,e,b,c,b b) a,e,a,d,b,c,a 
c) e,b,a,d,b,e d) c,b,d,a,e,c 
a b c 

d e 


2. Does each of these lists of vertices form a path in the 
following graph? Which paths are simple? Which are cir- 
cuits? What are the lengths of those that are paths? 


a) a,b,e,c,b b) a,d,a,d,a 

c) a,d,b,e,a d) a,b,e,c,b,d,a 
a b c 

d e 


In Exercises 3—5 determine whether the given graph is con- 
nected. 


AV 
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10. 


11. 


. How many connected components does each of the 


graphs in Exercises 3-5 have? For each graph find each 
of its connected components. 


. What do the connected components of acquaintanceship 


graphs represent? 


. What do the connected components of a collaboration 


graph represent? 


. Explain why in the collaboration graph of mathematicians 


(see Example 3 in Section 10.1) a vertex representing a 
mathematician is in the same connected component as the 
vertex representing Paul Erdés if and only if that mathe- 
matician has a finite Erd6s number. 


In the Hollywood graph (see Example 3 in Section 10.1), 
when is the vertex representing an actor in the same con- 
nected component as the vertex representing Kevin Ba- 
con? 


Determine whether each of these graphs is strongly con- 
nected and if not, whether it is weakly connected. 


a) a b c 
e d 
b) 2 b 
c 
e d 
c) 
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Determine whether each of these graphs is strongly con- 
nected and if not, whether it is weakly connected. 


a) a b c 


JF e d 

b) b 

a c 

if d 

ée 

c) a b c 

ef e 
What do the strongly connected components of a tele- 


phone call graph represent? 


Find the strongly connected components of each of these 
graphs. 
a) a b c 
e d 
b) a b c 
f e d 
c) 
a b c d e 
i h g Pa 


Find the strongly connected components of each of these 
graphs. 


a) in a 


b) 2 b (a d 
h g f e 
Cc) b 
i h g f 


Suppose that G = (V, E) is adirected graph. A vertex w € V 
is reachable from a vertex v € V if there is a directed path 
from v to w. The vertices v and w are mutually reachable if 
there are both a directed path from v to w and a directed path 
from w to vin G. 


16. 


17. 


18. 


19. 


20. 


21. 


Show that if G = (V, E) is a directed graph and u, v, and 
w are vertices in V for which uw and v are mutually reach- 
able and v and w are mutually reachable, then u and w 
are mutually reachable. 


Show that if G = (V, E) is a directed graph, then the 
strong components of two vertices u and v of V are either 
the same or disjoint. [Hint: Use Exercise 16.] 


Show that all vertices visited in a directed path connecting 
two vertices in the same strongly connected component 
of a directed graph are also in this strongly connected 
component. 


Find the number of paths of length n between two differ- 
ent vertices in Ky if n is 
a) 2. b) 3. 


c) 4. d) 5. 


Use paths either to show that these graphs are not isomor- 
phic or to find an isomorphism between these graphs. 


uy Uy vy Vo 
Ug uz V4 V3 

G H 
Use paths either to show that these graphs are not isomor- 
phic or to find an isomorphism between them. 


uy Uy vy Vo 


Ug U3 Vg V3 


uy Ug V7 V4 


22. Use paths either to show that these graphs are not isomor- 
phic or to find an isomorphism between them. 


uy Uy vy V9 

Ug U3 vg V3 

uz U4 V7 V4 
U6 Us V6 V5 


G vei 
23. Use paths either to show that these graphs are not isomor- 
phic or to find an isomorphism between them. 


24. Find the number of paths of length n between any two ad- 
jacent vertices in K3 3 for the values of n in Exercise 19. 

25. Find the number of paths of length n between any two 
nonadjacent vertices in K33 for the values of n in Exer- 
cise 19. 

26. Find the number of paths between c and d in the graph in 
Figure | of length 


a) 2. b) 3. c) 4. d) 5. e) 6. f) 7. 


27. Find the number of paths from a to e in the directed graph 
in Exercise 2 of length 


a) 2. b) 3. c) 4. d) 5. e) 6. f) 7. 
*28. Show that every connected graph with n vertices has at 


least n — 1 edges. 

29. Let G = (V, E) be a simple graph. Let R be the relation 
on V consisting of pairs of vertices (u, v) such that there 
is a path from u to v or such that u = v. Show that R is 
an equivalence relation. 

* 30. Show that in every simple graph there is a path from every 
vertex of odd degree to some other vertex of odd degree. 


In Exercises 3 1—33 find all the cut vertices of the given graph. 


31. a d e 32. 4 f 
, rs ; b ic d e 
33. a b f 
e 
c &§ 
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34. Find all the cut edges in the graphs in Exercises 31-33. 


* 35. Suppose that v is an endpoint of a cut edge. Prove that v 
is a cut vertex if and only if this vertex is not pendant. 


* 36. Show that a vertex c in the connected simple graph G is 
a cut vertex if and only if there are vertices u and v, both 
different from c, such that every path between wu and v 
passes through c. 


* 37. Show that a simple graph with at least two vertices has at 
least two vertices that are not cut vertices. 


* 38. Show that an edge in a simple graph is a cut edge if and 
only if this edge is not part of any simple circuit in the 
graph. 


39. A communications link in a network should be provided 
with a backup link if its failure makes it impossible for 
some message to be sent. For each of the communications 
networks shown here in (a) and (b), determine those links 
that should be backed up. 


a) Boston 


gaa New York 


San Francisco 


Washington 
Denver eames 


Los Angeles 
b) Bangor 
Burlington 
Seattle Portland 


Boston 


Denver Chicago 
New York 


Washington 


Atlanta 


Los City 
Angeles 


A vertex basis in a directed graph G is a minimal set B of 
vertices of G such that for each vertex v of G not in B there 
is a path to v from some vertex B. 


40. Find a vertex basis for each of the directed graphs in Ex- 
ercises 7—9 of Section 10.2. 


41. What is the significance of a vertex basis in an influence 
graph (described in Example 2 of Section 10.1)? Find a 
vertex basis in the influence graph in that example. 


42. Show that if a connected simple graph G is the union of 
the graphs G,; and Go, then G, and G2 have at least one 
common vertex. 


* 43, Show that if a simple graph G has k connected compo- 


nents and these components have n|, 72, ..., Nx vertices, 
respectively, then the number of edges of G does not ex- 
ceed 


k 
y Cue, 2). 
i=l 
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* 44, 


* 45, 


46. 


47. 


48. 


49. 


50. 


51. 


52. 
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Use Exercise 43 to show that a simple graph with n 
vertices and k connected components has at most (n — 
k)(n —k + 1)/2 edges. [Hint: First show that 


k 
2 2 

Yoni <n’ —(k—1)(2n—h), 

i=l 
where n; is the number of vertices in the ith connected 
component. ] 
Show that a simple graph G with n vertices is connected 
if it has more than (n — 1)(n — 2)/2 edges. 
Describe the adjacency matrix of a graph with n con- 
nected components when the vertices of the graph are 
listed so that vertices in each connected component are 
listed successively. 
How many nonisomorphic connected simple graphs are 
there with n vertices when n is 


a) 2? b) 3? c) 4? d) 5? 


Show that each of the following graphs has no cut ver- 
tices. 

a) C, where n > 3 

b) W, where n > 3 

c) Kim.» where m > 2 andn > 2 

d) QO, where n > 2 

Show that each of the graphs in Exercise 48 has no cut 
edges. 

For each of these graphs, find «(G), A(G), and 
minycy deg(v), and determine which of the two inequal- 
ities in«(G) < A(G) < minyey deg(v) are strict. 


a) @4 c 


e d 


Show that if G is a connected graph, then it is possible to 
remove vertices to disconnect G if and only if G is nota 
complete graph. 

Show that if G is aconnected graph with n vertices then 
a) «(G) =n — 1if and only if G = Ky. 

b) A(G) =n — 1 if and only if G = Kn. 


*55. 
56. 


53. 


54, 


57. 


58. 


59. 


60. 


61. 


62. 


63. 


64. 


Find k(Kjy),,) and A(Kin,n), Where m and n are positive 
integers. 


Construct a graph G with «(G) = 1, A(G) = 2, and 
Minycy deg(v) = 3. 


Show that if G is a graph, then k(G) < A(G). 


Explain how Theorem 2 can be used to find the length of 
the shortest path from a vertex v to a vertex w in a graph. 


Use Theorem 2 to find the length of the shortest path 
between a and f in the graph in Figure 1. 


Use Theorem 2 to find the length of the shortest path from 
a to c in the directed graph in Exercise 2. 


Let P; and Pz be two simple paths between the vertices u 
and v in the simple graph G that do not contain the same 
set of edges. Show that there is a simple circuit in G. 


Show that the existence of a simple circuit of length k, 
where k is an integer greater than 2, is an invariant for 
graph isomorphism. 


Explain how Theorem 2 can be used to determine whether 
a graph is connected. 


Use Exercise 61 to show that the graph G, in Figure 2 
is connected whereas the graph G2 in that figure is not 
connected. 


Show that a simple graph G is bipartite if and only if it 
has no circuits with an odd number of edges. 


In an old puzzle attributed to Alcuin of York (735-804), a 
farmer needs to carry a wolf, a goat, and a cabbage across 
ariver. The farmer only has a small boat, which can carry 
the farmer and only one object (an animal or a vegetable). 
He can cross the river repeatedly. However, if the farmer 
is on the other shore, the wolf will eat the goat, and, simi- 
larly, the goat will eat the cabbage. We can describe each 
state by listing what is on each shore. For example, we can 
use the pair (FG, WC ) for the state where the farmer and 
goat are on the first shore and the wolf and cabbage are 
on the other shore. [The symbol % is used when nothing 
is on a shore, so that (FWGC, @) is the initial state. ] 
a) Find all allowable states of the puzzle, where neither 
the wolf and the goat nor the goat and the cabbage are 
left on the same shore without the farmer. 


b) Construct a graph such that each vertex of this graph 
represents an allowable state and the vertices repre- 
senting two allowable states are connected by an edge 
if it is possible to move from one state to the other us- 
ing one trip of the boat. 


c) Explain why finding a path from the vertex represent- 
ing (FWGC, ) to the vertex representing (J, FWGC ) 
solves the puzzle. 


d) Find two different solutions of the puzzle, each using 
seven crossings. 

e) Suppose that the farmer must pay a toll of one dollar 
whenever he crosses the river with an animal. Which 
solution of the puzzle should the farmer use to pay the 
least total toll? 


*65. Use a graph model and a path in your graph, as in Exer- 


cise 64, to solve the jealous husbands problem. Two 
married couples, each a husband and a wife, want to 
cross a river. They can only use a boat that can carry 
one or two people from one shore to the other shore. 
Each husband is extremely jealous and is not willing 
to leave his wife with the other husband, either in the 
boat or on shore. How can these four people reach the 


opposite shore? 
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66. Suppose that you have a three-gallon jug and a five-gallon 


jug. You may fill either jug with water, you may empty 
either jug, and you may transfer water from either jug 
into the other jug. Use a path in a directed graph to 
show that you can end up with a jug containing exactly 
one gallon. [Hint: Use an ordered pair (a, b) to indicate 
how much water is in each jug. Represent these ordered 
pairs by vertices. Add an edge for each allowable opera- 
tion with the jugs.] 


Introduction 


Can we travel along the edges of a graph starting at a vertex and returning to it by traversing 
each edge of the graph exactly once? Similarly, can we travel along the edges of a graph starting 
at a vertex and returning to it while visiting each vertex of the graph exactly once? Although 
these questions seem to be similar, the first question, which asks whether a graph has an Euler 
circuit, can be easily answered simply by examining the degrees of the vertices of the graph, 
while the second question, which asks whether a graph has a Hamilton circuit, is quite difficult 
to solve for most graphs. In this section we will study these questions and discuss the difficulty 
of solving them. Although both questions have many practical applications in many different 
areas, both arose in old puzzles. We will learn about these old puzzles as well as modern 


practical applications. 


Euler Paths and Circuits 


The town of K6nigsberg, Prussia (now called Kaliningrad and part of the Russian republic), 
was divided into four sections by the branches of the Pregel River. These four sections included 


Links 


Only five bridges connect 
Kaliningrad today. Of 
these, just two remain 
from Euler’s day. 


the two regions on the banks of the Pregel, Kneiphof Island, and the region between the two 
branches of the Pregel. In the eighteenth century seven bridges connected these regions. Figure | 
depicts these regions and bridges. 

The townspeople took long walks through town on Sundays. They wondered whether it was 
possible to start at some location in the town, travel across all the bridges once without crossing 
any bridge twice, and return to the starting point. 


The Swiss mathematician Leonhard Euler solved this problem. His solution, published 
in 1736, may be the first use of graph theory. (For a translation of Euler’s original paper see 
[BiLIWi199].) Euler studied this problem using the multigraph obtained when the four regions 
are represented by vertices and the bridges by edges. This multigraph is shown in Figure 2. 


=) 
oo 


FIGURE 1 The Seven Bridges of Kénigsberg. 


Cc 
> 
B 


FIGURE 2 Multigraph Model 
of the Town of K6nigsberg. 
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The problem of traveling across every bridge without crossing any bridge more than once 
can be rephrased in terms of this model. The question becomes: Is there a simple circuit in this 
multigraph that contains every edge? 


DEFINITION 1 — An Euler circuit in a graph G is a simple circuit containing every edge of G. An Euler path 
in G is a simple path containing every edge of G. 


Examples | and 2 illustrate the concept of Euler circuits and paths. 


EXAMPLE1_ Which of the undirected graphs in Figure 3 have an Euler circuit? Of those that do not, which 
have an Euler path? 


Solution: The graph G; has an Euler circuit, for example, a, e,c,d,e,b,a. Neither of the 
graphs G2 or G3 has an Euler circuit (the reader should verify this). However, G3 has 
an Euler path, namely, a,c,d,e,b,d,a,b. G2 does not have an Euler path (as the reader 
should verify). < 


EXAMPLE 2 Which of the directed graphs in Figure 4 have an Euler circuit? Of those that do not, which have 
an Euler path? 


Extra i Solution: The graph H2 has an Euler circuit, for example, a, g,c, b, g,e,d, f,a. Neither Hi 
Examples MA nor H3 has an Euler circuit (as the reader should verify). H3 has an Euler path, namely, 
c,a, b,c, d,b, but H; does not (as the reader should verify). 


NECESSARY AND SUFFICIENT CONDITIONS FOR EULER CIRCUITS AND PATHS 
There are simple criteria for determining whether a multigraph has an Euler circuit or an Euler 
path. Euler discovered them when he solved the famous Konigsberg bridge problem. We will 
assume that all graphs discussed in this section have a finite number of vertices and edges. 

What can we say if a connected multigraph has an Euler circuit? What we can show is that 
every vertex must have even degree. To do this, first note that an Euler circuit begins with a 
vertex a and continues with an edge incident with a, say {a, b}. The edge {a, b} contributes one 
to deg(a). Each time the circuit passes through a vertex it contributes two to the vertex’s degree, 
because the circuit enters via an edge incident with this vertex and leaves via another such edge. 
Finally, the circuit terminates where it started, contributing one to deg(a). Therefore, deg(a) 
must be even, because the circuit contributes one when it begins, one when it ends, and two 
every time it passes through a (if it ever does). A vertex other than a has even degree because 
the circuit contributes two to its degree each time it passes through the vertex. We conclude that 
if a connected graph has an Euler circuit, then every vertex must have even degree. 

Is this necessary condition for the existence of an Euler circuit also sufficient? That is, must 
an Euler circuit exist in a connected multigraph if all vertices have even degree? This question 
can be settled affirmatively with a construction. 


a b 
a b a b a b a b ; j c d 
= | NA | NI 
d c d é (a d e d c i a b 
e d 
Hy, Hy A 


rea ron ig. 


FIGURE 3 The Undirected Graphs G1, G2, and G3. FIGURE 4 The Directed Graphs Hj, H>, and H3. 


10.5 Euler and Hamilton Paths 695 


G H 


FIGURE 5 Constructing an Euler Circuit in G. 


Suppose that G is a connected multigraph with at least two vertices and the degree of 

every vertex of G is even. We will form a simple circuit that begins at an arbitrary vertex a 
© of G, building it edge by edge. Let x9 = a. First, we arbitrarily choose an edge {xg, x;} inci- 

dent with a which is possible because G is connected. We continue by building a simple path 
{xo, x1}, {x1, X2},.-.., {Xn—1, Xn}, successively adding edges one by one to the path until we 
cannot add another edge to the path. This happens when we reach a vertex for which we have 
already included all edges incident with that vertex in the path. For instance, in the graph G in 
Figure 5 we begin at a and choose in succession the edges {a, f}, {f, c}, {c, b}, and {b, a}. 

The path we have constructed must terminate because the graph has a finite number of 
edges, so we are guaranteed to eventually reach a vertex for which no edges are available to add 
to the path. The path begins at a with an edge of the form {a, x}, and we now show that it must 
terminate at a with an edge of the form {y, a}. To see that the path must terminate at a, note that 
each time the path goes through a vertex with even degree, it uses only one edge to enter this 
vertex, so because the degree must be at least two, at least one edge remains for the path to leave 
the vertex. Furthermore, every time we enter and leave a vertex of even degree, there are an even 
number of edges incident with this vertex that we have not yet used in our path. Consequently, 
as we form the path, every time we enter a vertex other than a, we can leave it. This means that 
the path can end only at a. Next, note that the path we have constructed may use all the edges 
of the graph, or it may not if we have returned to a for the last time before using all the edges. 

An Euler circuit has been constructed if all the edges have been used. Otherwise, consider 
the subgraph H obtained from G by deleting the edges already used and vertices that are not 
incident with any remaining edges. When we delete the circuit a, f, c, b, a from the graph in 
Figure 5, we obtain the subgraph labeled as H. 

Because G is connected, H has at least one vertex in common with the circuit that has been 
deleted. Let w be such a vertex. (In our example, c is the vertex.) 


LEONHARD EULER (1707-1783) Leonhard Euler was the son of a Calvinist minister from the vicinity of 
Basel, Switzerland. At 13 he entered the University of Basel, pursuing a career in theology, as his father wished. 
At the university Euler was tutored by Johann Bernoulli of the famous Bernoulli family of mathematicians. 
His interest and skills led him to abandon his theological studies and take up mathematics. Euler obtained his 
master’s degree in philosophy at the age of 16. In 1727 Peter the Great invited him to join the Academy at 
St. Petersburg. In 1741 he moved to the Berlin Academy, where he stayed until 1766. He then returned to St. 
Petersburg, where he remained for the rest of his life. 

Euler was incredibly prolific, contributing to many areas of mathematics, including number theory, com- 
binatorics, and analysis, as well as its applications to such areas as music and naval architecture. He wrote 
over 1100 books and papers and left so much unpublished work that it took 47 years after he died for all his work to be published. 
During his life his papers accumulated so quickly that he kept a large pile of articles awaiting publication. The Berlin Academy 
published the papers on top of this pile so later results were often published before results they depended on or superseded. Euler 
had 13 children and was able to continue his work while a child or two bounced on his knees. He was blind for the last 17 years of 
his life, but because of his fantastic memory this did not diminish his mathematical output. The project of publishing his collected 
works, undertaken by the Swiss Society of Natural Science, is ongoing and will require more than 75 volumes. 
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Every vertex in H has even degree (because in G all vertices had even degree, and for each 
vertex, pairs of edges incident with this vertex have been deleted to form H). Note that H may 
not be connected. Beginning at w, construct a simple path in H by choosing edges as long as 
possible, as was done in G. This path must terminate at w. For instance, in Figure 5, c, d, e, c 1s 
a path in H. Next, form a circuit in G by splicing the circuit in H with the original circuit in G 
(this can be done because w is one of the vertices in this circuit). When this is done in the graph 
in Figure 5, we obtain the circuit a, f, c,d, e, c, b, a. 

Continue this process until all edges have been used. (The process must terminate because 
there are only a finite number of edges in the graph.) This produces an Euler circuit. The 
construction shows that if the vertices of a connected multigraph all have even degree, then the 
graph has an Euler circuit. 

We summarize these results in Theorem 1. 


A connected multigraph with at least two vertices has an Euler circuit if and only if each of 
its vertices has even degree. 


We can now solve the K6nigsberg bridge problem. Because the multigraph representing 
these bridges, shown in Figure 2, has four vertices of odd degree, it does not have an Euler 
circuit. There is no way to start at a given point, cross each bridge exactly once, and return to 
the starting point. 

Algorithm | gives the constructive procedure for finding Euler circuits given in the discus- 
sion preceding Theorem |. (Because the circuits in the procedure are chosen arbitrarily, there 
is some ambiguity. We will not bother to remove this ambiguity by specifying the steps of the 
procedure more precisely.) 


ALGORITHM 1 Constructing Euler Circuits. 


procedure Euler(G: connected multigraph with all vertices of 
even degree) 
circuit := a circuit in G beginning at an arbitrarily chosen 
vertex with edges successively added to form a path that 
returns to this vertex 
H := G with the edges of this circuit removed 
while H has edges 
subcircuit := a circuit in H beginning at a vertex in H that 
also is an endpoint of an edge of circuit 
H := H with edges of subcircuit and all isolated vertices 
removed 
circuit := circuit with subcircuit inserted at the appropriate 
vertex 
return circuit {circuit is an Euler circuit} 


Algorithm | provides an efficient algorithm for finding Euler circuits in a connected multi- 
graph G with all vertices of even degree. We leave it to the reader (Exercise 66) to show that 
the worst case complexity of this algorithm is O(m), where m is the number of edges of G. 

Example 3 shows how Euler paths and circuits can be used to solve a type of puzzle. 


EXAMPLE 3 


THEOREM 2 


EXAMPLE 4 
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FIGURE 6 Mohammed’s Scimitars. 


Many puzzles ask you to draw a picture in a continuous motion without lifting a pencil so that 
no part of the picture is retraced. We can solve such puzzles using Euler circuits and paths. 
For example, can Mohammed’s scimitars, shown in Figure 6, be drawn in this way, where the 
drawing begins and ends at the same point? 


Solution: We can solve this problem because the graph G shown in Figure 6 has an Euler circuit. 
It has such a circuit because all its vertices have even degree. We will use Algorithm | to construct 
an Euler circuit. First, we form the circuit a, b,d,c, b, e,i, f, e, a. We obtain the subgraph H 
by deleting the edges in this circuit and all vertices that become isolated when these edges are 
removed. Then we form the circuit d, g,h, j,i,h,k, g, f,d in H. After forming this circuit we 
have used all edges in G. Splicing this new circuit into the first circuit at the appropriate place 
produces the Euler circuit a, b,d, g,h, j,i,h,k, g, f,d,c,b,e,i, f, e, a. This circuit gives a 
way to draw the scimitars without lifting the pencil or retracing part of the picture. < 


Another algorithm for constructing Euler circuits, called Fleury’s algorithm, is described in 
the premble to Exercise 50. 

We will now show that a connected multigraph has an Euler path (and not an Euler circuit) if 
and only if it has exactly two vertices of odd degree. First, suppose that a connected multigraph 
does have an Euler path from a to b, but not an Euler circuit. The first edge of the path contributes 
one to the degree of a. A contribution of two to the degree of a is made every time the path 
passes through a. The last edge in the path contributes one to the degree of b. Every time the 
path goes through b there is a contribution of two to its degree. Consequently, both a and b have 
odd degree. Every other vertex has even degree, because the path contributes two to the degree 
of a vertex whenever it passes through it. 

Now consider the converse. Suppose that a graph has exactly two vertices of odd degree, 
say a and b. Consider the larger graph made up of the original graph with the addition of an 
edge {a, b}. Every vertex of this larger graph has even degree, so there is an Euler circuit. The 
removal of the new edge produces an Euler path in the original graph. Theorem 2 summarizes 
these results. 


A connected multigraph has an Euler path but not an Euler circuit if and only if it has exactly 
two vertices of odd degree. 


Which graphs shown in Figure 7 have an Euler path? 


Solution: G1 contains exactly two vertices of odd degree, namely, b and d. Hence, it has an Euler 
path that must have and d as its endpoints. One such Euler path is d, a, b, c, d, b. Similarly, G2 
has exactly two vertices of odd degree, namely, b and d. So it has an Euler path that must have 
b and d as endpoints. One such Euler path is b, a, g, f,e,d,c, g,b,c, f,d. G3 has no Euler 
path because it has six vertices of odd degree. < 


Returning to eighteenth-century K6nigsberg, is it possible to start at some point in the 
town, travel across all the bridges, and end up at some other point in town? This question can 
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DEFINITION 2 
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1 2 3 


FIGURE 7 Three Undirected Graphs. 


be answered by determining whether there is an Euler path in the multigraph representing the 
bridges in K6nigsberg. Because there are four vertices of odd degree in this multigraph, there 
is no Euler path, so such a trip is impossible. 

Necessary and sufficient conditions for Euler paths and circuits in directed graphs are given 
in Exercises 16 and 17. 


APPLICATIONS OF EULER PATHS AND CIRCUITS Euler paths and circuits can be used 
to solve many practical problems. For example, many applications ask for a path or circuit that 
traverses each street in a neighborhood, each road in a transportation network, each connection 
in a utility grid, or each link in a communications network exactly once. Finding an Euler path 
or circuit in the appropriate graph model can solve such problems. For example, if a postman 
can find an Euler path in the graph that represents the streets the postman needs to cover, this 
path produces a route that traverses each street of the route exactly once. If no Euler path exists, 
some streets will have to be traversed more than once. The problem of finding a circuit in a graph 
with the fewest edges that traverses every edge at least once is known as the Chinese postman 
problem in honor of Guan Meigu, who posed it in 1962. See [MiRo91] for more information 
on the solution of the Chinese postman problem when no Euler path exists. 

Among the other areas where Euler circuits and paths are applied is in the layout of circuits, 
in network multicasting, and in molecular biology, where Euler paths are used in the sequencing 
of DNA. 


Hamilton Paths and Circuits 


We have developed necessary and sufficient conditions for the existence of paths and circuits 
that contain every edge of a multigraph exactly once. Can we do the same for simple paths and 
circuits that contain every vertex of the graph exactly once? 


A simple path in a graph G that passes through every vertex exactly once is called a Hamilton 
path, and a simple circuit in a graph G that passes through every vertex exactly once is called 


a Hamilton circuit. That is, the simple path xo, x1, ..., Xn—1, Xn inthe graph G = (V, E) isa 
Hamilton path if V = {xo, x1, ...,Xn—-1, Xn} and x; A x; forO <i < j <n, and the simple 
circuit x9, X1,..-,Xn—1,Xn, Xo (with n > 0) is a Hamilton circuit if x9, x1, ..., Xn—1, Xn IS 


a Hamilton path. 


This terminology comes from a game, called the Icosian puzzle, invented in 1857 by the 
Trish mathematician Sir William Rowan Hamilton. It consisted of a wooden dodecahedron [a 
polyhedron with 12 regular pentagons as faces, as shown in Figure 8(a)], with a peg at each 
vertex of the dodecahedron, and string. The 20 vertices of the dodecahedron were labeled with 
different cities in the world. The object of the puzzle was to start at a city and travel along the 
edges of the dodecahedron, visiting each of the other 19 cities exactly once, and end back at the 
first city. The circuit traveled was marked off using the string and pegs. 


EXAMPLE 5 


Extra 
Examples 


~~ 
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(a) (b) 
FIGURE 9 A Solution to 
FIGURE 8 Hamilton’s “A Voyage Round the the “A Voyage Round the 
World” Puzzle. World” Puzzle. 


Because the author cannot supply each reader with a wooden solid with pegs and string, we 
will consider the equivalent question: Is there a circuit in the graph shown in Figure 8(b) that 
passes through each vertex exactly once? This solves the puzzle because this graph is isomorphic 
to the graph consisting of the vertices and edges of the dodecahedron. A solution of Hamilton’s 
puzzle is shown in Figure 9. 


Which of the simple graphs in Figure 10 have a Hamilton circuit or, if not, a Hamilton path? 


Solution: G, has a Hamilton circuit: a, b, c, d, e, a. There is no Hamilton circuit in G2 (this can 
be seen by noting that any circuit containing every vertex must contain the edge {a, b} twice), 
but G2 does have a Hamilton path, namely, a, b, c, d. G3 has neither a Hamilton circuit nor a 
Hamilton path, because any path containing all vertices must contain one of the edges {a, b}, 
{e, f}, and {c, d} more than once. 
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FIGURE 10 Three Simple Graphs. 


CONDITIONS FORTHE EXISTENCE OF HAMILTON CIRCUITS Is there a simple way 
to determine whether a graph has a Hamilton circuit or path? At first, it might seem that there 
should be an easy way to determine this, because there is a simple way to answer the similar 
question of whether a graph has an Euler circuit. Surprisingly, there are no known simple 
necessary and sufficient criteria for the existence of Hamilton circuits. However, many theorems 
are known that give sufficient conditions for the existence of Hamilton circuits. Also, certain 
properties can be used to show that a graph has no Hamilton circuit. For instance, a graph with a 
vertex of degree one cannot have a Hamilton circuit, because in a Hamilton circuit, each vertex 
is incident with two edges in the circuit. Moreover, if a vertex in the graph has degree two, then 
both edges that are incident with this vertex must be part of any Hamilton circuit. Also, note 
that when a Hamilton circuit is being constructed and this circuit has passed through a vertex, 
then all remaining edges incident with this vertex, other than the two used in the circuit, can be 
removed from consideration. Furthermore, a Hamilton circuit cannot contain a smaller circuit 
within it. 
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FIGURE 11 Two Graphs That Do Not Have a Hamilton Circuit. 


EXAMPLE 6 — Show that neither graph displayed in Figure 11 has a Hamilton circuit. 


Solution: There is no Hamilton circuit in G because G has a vertex of degree one, namely, e. 
Now consider H. Because the degrees of the vertices a, b, d, and e are all two, every edge 
incident with these vertices must be part of any Hamilton circuit. It is now easy to see that no 
Hamilton circuit can exist in H, for any Hamilton circuit would have to contain four edges 
incident with c, which is impossible. <{ 


EXAMPLE 7 — Show that K,, has a Hamilton circuit whenever n > 3. 


Solution: We can form a Hamilton circuit in K, beginning at any vertex. Such a circuit can be 
built by visiting vertices in any order we choose, as long as the path begins and ends at the same 
vertex and visits each other vertex exactly once. This is possible because there are edges in K, 
between any two vertices. < 


Although no useful necessary and sufficient conditions for the existence of Hamilton circuits 
are known, quite a few sufficient conditions have been found. Note that the more edges a graph 
has, the more likely it is to have a Hamilton circuit. Furthermore, adding edges (but not vertices) 
to a graph with a Hamilton circuit produces a graph with the same Hamilton circuit. So as we 
add edges to a graph, especially when we make sure to add edges to each vertex, we make it 


WILLIAM ROWAN HAMILTON (1805-1865) William Rowan Hamilton, the most famous Irish scien- 
tist ever to have lived, was born in 1805 in Dublin. His father was a successful lawyer, his mother came 
from a family noted for their intelligence, and he was a child prodigy. By the age of 3 he was an excel- 
lent reader and had mastered advanced arithmetic. Because of his brilliance, he was sent off to live with 
his uncle James, a noted linguist. By age 8 Hamilton had learned Latin, Greek, and Hebrew; by 10 he had 
also learned Italian and French and he began his study of oriental languages, including Arabic, Sanskrit, and 
Persian. During this period he took pride in knowing as many languages as his age. At 17, no longer de- 
voted to learning new languages and having mastered calculus and much mathematical astronomy, he began 
original work in optics, and he also found an important mistake in Laplace’s work on celestial mechanics. 
Before entering Trinity College, Dublin, at 18, Hamilton had not attended school; rather, he received private tutoring. At Trinity, he 
was a superior student in both the sciences and the classics. Prior to receiving his degree, because of his brilliance he was appointed 
the Astronomer Royal of Ireland, beating out several famous astronomers for the post. He held this position until his death, living 
and working at Dunsink Observatory outside of Dublin. Hamilton made important contributions to optics, abstract algebra, and 
dynamics. Hamilton invented algebraic objects called quaternions as an example of a noncommutative system. He discovered the 
appropriate way to multiply quaternions while walking along a canal in Dublin. In his excitement, he carved the formula in the stone 
of a bridge crossing the canal, a spot marked today by a plaque. Later, Hamilton remained obsessed with quaternions, working to 
apply them to other areas of mathematics, instead of moving to new areas of research. 

In 1857 Hamilton invented “The Icosian Game” based on his work in noncommutative algebra. He sold the idea for 25 pounds 
to a dealer in games and puzzles. (Because the game never sold well, this turned out to be a bad investment for the dealer.) The 
“Traveler’s Dodecahedron,” also called “A Voyage Round the World,” the puzzle described in this section, is a variant of that game. 

Hamilton married his third love in 1833, but his marriage worked out poorly, because his wife, a semi-invalid, was unable to 
cope with his household affairs. He suffered from alcoholism and lived reclusively for the last two decades of his life. He died from 
gout in 1865, leaving masses of papers containing unpublished research. Mixed in with these papers were a large number of dinner 
plates, many containing the remains of desiccated, uneaten chops. 
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increasingly likely that a Hamilton circuit exists in this graph. Consequently, we would expect 
there to be sufficient conditions for the existence of Hamilton circuits that depend on the degrees 
of vertices being sufficiently large. We state two of the most important sufficient conditions here. 
These conditions were found by Gabriel A. Dirac in 1952 and @ystein Ore in 1960. 


DIRAC’S THEOREM If G is a simple graph with n vertices with n > 3 such that the 
degree of every vertex in G is at least n/2, then G has a Hamilton circuit. 


ORE’S THEOREM If G is a simple graph with n vertices with n >3 such that 
deg(u) + deg(v) > n for every pair of nonadjacent vertices u and v in G, then G has a 
Hamilton circuit. 


The proof of Ore’s theorem is outlined in Exercise 65. Dirac’s theorem can be proved as a 
corollary to Ore’s theorem because the conditions of Dirac’s theorem imply those of Ore’s 
theorem. 

Both Ore’s theorem and Dirac’s theorem provide sufficient conditions for a connected simple 
graph to have a Hamilton circuit. However, these theorems do not provide necessary conditions 
for the existence of a Hamilton circuit. For example, the graph C5 has a Hamilton circuit but 
does not satisfy the hypotheses of either Ore’s theorem or Dirac’s theorem, as the reader can 
verify. 


The best algorithms known for finding a Hamilton circuit in a graph or determining that 
no such circuit exists have exponential worst-case time complexity (in the number of vertices 
of the graph). Finding an algorithm that solves this problem with polynomial worst-case time 


GABRIEL ANDREW DIRAC (1925-1984) Gabriel Dirac was born in Budapest. He moved to England in 
1937 when his mother married the famous physicist and Nobel Laureate Paul Adrien Maurice Dirac, who 
adopted him. Gabriel A. Dirac entered Cambridge University in 1942, but his studies were interrupted by 
wartime service in the aviation industry. He obtained his Ph.D. in mathematics in 1951 from the University of 
London. He held university positions in England, Canada, Austria, Germany, and Denmark, where he spent his 
last 14 years. Dirac became interested in graph theory early in his career and help raise its status as an important 
topic of research. He made important contributions to many aspects of graph theory, including graph coloring 
and Hamilton circuits. Dirac attracted many students to graph theory and was noted as an excellent lecturer. 
Dirac was noted for his penetrating mind and held unconventional views on many topics, including politics 


and social life. Dirac was a man with many interests and held a great passion for fine art. He had a happy family life with his wife 
Rosemari and his four children. 


@Y STEIN ORE (1899-1968) Ore was born in Kristiania (the old name for Oslo, Norway). In 1922 he received 
his bachelors degree and in 1925 his Ph.D. in mathematics from Kristiania University, after studies in Germany 
and in Sweden. In 1927 he was recruited to leave his junior position at Kristiania and join Yale University. He 
was promoted rapidly at Yale, becoming full professor in 1929 and Sterling Professor in 1931, a position he 
held until 1968. 

Ore made many contributions to number theory, ring theory, lattice theory, graph theory, and probability 
theory. He was a prolific author of papers and books. His interest in the history of mathematics is reflected in 
his biographies of Abel and Cardano, and in his popular textbook Number Theory and its History. He wrote 
four books on graph theory in the 1960s. 


During and after World War II Ore played a major role supporting his native Norway. In 1947 King Haakon VII of Norway 
gave him the Knight Order of St. Olaf to recognize these efforts. Ore possessed deep knowledge of painting and sculpture and was 
an ardent collector of ancient maps. He was married and had two children. 
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Links 


complexity would be a major accomplishment because it has been shown that this problem 
is NP-complete (see Section 3.3). Consequently, the existence of such an algorithm would 
imply that many other seemingly intractable problems could be solved using algorithms with 
polynomial worst-case time complexity. 


Applications of Hamilton Circuits 


Hamilton paths and circuits can be used to solve practical problems. For example, many appli- 
cations ask for a path or circuit that visits each road intersection in a city, each place pipelines 
intersect in a utility grid, or each node in a communications network exactly once. Finding a 
Hamilton path or circuit in the appropriate graph model can solve such problems. The famous 
traveling salesperson problem or TSP (also known in older literature as the traveling sales- 
man problem) asks for the shortest route a traveling salesperson should take to visit a set of 
cities. This problem reduces to finding a Hamilton circuit in a complete graph such that the total 
weight of its edges is as small as possible. We will return to this question in Section 10.6. 
We now describe a less obvious application of Hamilton circuits to coding. 


Gray Codes The position of a rotating pointer can be represented in digital form. One way to 
do this is to split the circle into 2” arcs of equal length and to assign a bit string of length n to 
each arc. Two ways to do this using bit strings of length three are shown in Figure 12. 

The digital representation of the position of the pointer can be determined using a set of n 
contacts. Each contact is used to read one bit in the digital representation of the position. This 
is illustrated in Figure 13 for the two assignments from Figure 12. 

When the pointer is near the boundary of two arcs, a mistake may be made in reading its 
position. This may result in a major error in the bit string read. For instance, in the coding 
scheme in Figure 12(a), if a small error is made in determining the position of the pointer, the 
bit string 100 is read instead of 011. All three bits are incorrect! To minimize the effect of an 
error in determining the position of the pointer, the assignment of the bit strings to the 2” arcs 
should be made so that only one bit is different in the bit strings represented by adjacent arcs. 
This is exactly the situation in the coding scheme in Figure 12(b). An error in determining the 
position of the pointer gives the bit string 010 instead of 011. Only one bit is wrong. 

A Gray code is a labeling of the arcs of the circle such that adjacent arcs are labeled with bit 
strings that differ in exactly one bit. The assignment in Figure 12(b) is a Gray code. We can find 
a Gray code by listing all bit strings of length n in such a way that each string differs in exactly 
one position from the preceding bit string, and the last string differs from the first in exactly one 
position. We can model this problem using the n-cube Q,,. What is needed to solve this problem 
is a Hamilton circuit in Q,. Such Hamilton circuits are easily found. For instance, a Hamilton 


FIGURE 12 Converting the Position of a Pointer into Digital Form. 
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FIGURE 13 The Digital Representation of the 
Position of the Pointer. 
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FIGURE 14 A Hamilton 
Circuit for Q3. 


circuit for Q3 is displayed in Figure 14. The sequence of bit strings differing in exactly one bit 
produced by this Hamilton circuit is 000, 001, 011, 010, 110, 111, 101, 100. 

Gray codes are named after Frank Gray, who invented them in the 1940s at AT&T Bell 
Laboratories to minimize the effect of errors in transmitting digital signals. < 


Exercises 


In Exercises 1-8 determine whether the given graph has an 


3. 4 b 
Euler circuit. Construct such a circuit when one exists. If 
no Euler circuit exists, determine whether the graph has an 
Euler path and construct such a path if one exists. 
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. Suppose that in addition to the seven bridges of KGnigs- 


berg (shown in Figure 1) there were two additional 
bridges, connecting regions B and C and regions B and 
D, respectively. Could someone cross all nine of these 
bridges exactly once and return to the starting point? 


Can someone cross all the bridges shown in this map ex- 
actly once and return to the starting point? 


When can the centerlines of the streets in a city be painted 
without traveling a street more than once? (Assume that 
all the streets are two-way streets.) 


Devise a procedure, similar to Algorithm 1, for construct- 
ing Euler paths in multigraphs. 


In Exercises 13-15 determine whether the picture shown can 
be drawn with a pencil in a continuous motion without lifting 
the pencil or retracing part of the picture. 


13. 


—- 


15. 


*16. 


*17, 


Show that a directed multigraph having no isolated ver- 
tices has an Euler circuit if and only if the graph is weakly 
connected and the in-degree and out-degree of each vertex 
are equal. 


Show that a directed multigraph having no isolated ver- 
tices has an Euler path but not an Euler circuit if and 
only if the graph is weakly connected and the in-degree 
and out-degree of each vertex are equal for all but two 
vertices, one that has in-degree one larger than its out- 
degree and the other that has out-degree one larger than 
its in-degree. 


In Exercises 18—23 determine whether the directed graph 
shown has an Euler circuit. Construct an Euler circuit if one 
exists. If no Euler circuit exists, determine whether the di- 
rected graph has an Euler path. Construct an Euler path if one 
exists. 


18. 


20. 


21. 


a 


19. a b 
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*24, 


25. 


26. 


27. 


28. 


29. 


) a rs 


Devise an algorithm for constructing Euler circuits in di- 
rected graphs. 


Devise an algorithm for constructing Euler paths in di- 
rected graphs. 


For which values of n do these graphs have an Euler cir- 
cuit? 
a) Ky, b) C, 


c) Wa d) Qn 


For which values of n do the graphs in Exercise 26 have 
an Euler path but no Euler circuit? 


For which values of m and n does the complete bipartite 
graph K,,,, have an 


a) Euler circuit? 
b) Euler path? 


Find the least number of times it is necessary to lift a 
pencil from the paper when drawing each of the graphs 
in Exercises 1—7 without retracing any part of the graph. 


In Exercises 30-36 determine whether the given graph has a 
Hamilton circuit. If it does, find such a circuit. If it does not, 
give an argument to show why no such circuit exists. 


30. 


31. 


33. 
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35. 
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37. 
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39. 


40. 


*41, 


42. 


43. 


44, 


45. 
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Does the graph in Exercise 30 have a Hamilton path? If 


so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 31 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 32 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 33 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 34 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 35 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 36 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


For which values of n do the graphs in Exercise 26 have 
a Hamilton circuit? 


For which values of m and n does the complete bipartite 
graph Kj, have a Hamilton circuit? 
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* 46. Show that the Petersen graph, shown here, does not have 48. Can you find a simple graph with n vertices with n > 3 
a Hamilton circuit, but that the subgraph obtained by that does not have a Hamilton circuit, yet the degree of 
deleting a vertex v, and all edges incident with v, does every vertex in the graph is at least (n — 1)/2? 


have a Hamilton circuit. . ; 
* 49, Show that there is a Gray code of order n whenever n is 


a a positive integer, or equivalently, show that the n-cube 
Q,,n > 1, always has a Hamilton circuit. [Hint: Use 
mathematical induction. Show how to produce a Gray 

A code of order n from one of order n — 1.] 


PL Fleury s algorithm, published in 1883, constructs Euler cir- 
4 cuits by first choosing an arbitrary vertex of a connected multi- 


> graph, and then forming a circuit by choosing edges succes- 
sively. Once an edge is chosen, it is removed. Edges are cho- 


sen successively so that each edge begins where the last edge 


d e ends, and so that this edge is not a cut edge unless there is no 
alternative. 
47. For each of these graphs, determine (i) whether Dirac’s 
theorem can be used to show that the graph has a Hamilton 50. Use Fleury’s algorithm to find an Euler circuit in the graph 
circuit, (i) whether Ore’s theorem can be used to show G in Figure 5. 


that the graph has a Hamilton circuit, and (iii) whether 
the graph has a Hamilton circuit. 


a) b) ## 52, Prove that Fleury’s algorithm always produces an Euler 
4 : circuit. 


* 53. Give a variant of Fleury’s algorithm to produce Euler 
paths. 


* 51. Express Fleury’s algorithm in pseudocode. 


54. A diagnostic message can be sent out over a computer 
o_o network to perform tests over all links and in all devices. 
c) d) What sort of paths should be used to test all links? To test 
all devices? 


55. Show that a bipartite graph with an odd number of vertices 
does not have a Hamilton circuit. 


JULIUS PETER CHRISTIAN PETERSEN (1839-1910) Julius Petersen was born in the Danish town of 
Sorg. His father was a dyer. In 1854 his parents were no longer able to pay for his schooling, so he became an 
apprentice in an uncle’s grocery store. When this uncle died, he left Petersen enough money to return to school. 
After graduating, he began studying engineering at the Polytechnical School in Copenhagen, later deciding to 
concentrate on mathematics. He published his first textbook, a book on logarithms, in 1858. When his inheritance 
ran out, he had to teach to make a living. From 1859 until 1871 Petersen taught at a prestigious private high 
school in Copenhagen. While teaching high school he continued his studies, entering Copenhagen University 
in 1862. He married Laura Bertelsen in 1862; they had three children, two sons and a daughter. 

Petersen obtained a mathematics degree from Copenhagen University in 1866 and finally obtained his 
doctorate in 1871 from that school. After receiving his doctorate, he taught at a polytechnic and military academy. In 1887 he was 
appointed to a professorship at the University of Copenhagen. Petersen was well known in Denmark as the author of a large series 
of textbooks for high schools and universities. One of his books, Methods and Theories for the Solution of Problems of Geometrical 
Construction, was translated into eight languages, with the English language version last reprinted in 1960 and the French version 
reprinted as recently as 1990, more than a century after the original publication date. 

Petersen worked in a wide range of areas, including algebra, analysis, cryptography, geometry, mechanics, mathematical 
economics, and number theory. His contributions to graph theory, including results on regular graphs, are his best-known work. 
He was noted for his clarity of exposition, problem-solving skills, originality, sense of humor, vigor, and teaching. One interesting 
fact about Petersen was that he preferred not to read the writings of other mathematicians. This led him often to rediscover results 
already proved by others, often with embarrassing consequences. However, he was often angry when other mathematicians did not 
read his writings! 

Petersen’s death was front-page news in Copenhagen. A newspaper of the time described him as the Hans Christian Andersen 
of science—a child of the people who made good in the academic world. 


A knight is a chess piece that can move either two spaces 
horizontally and one space vertically or one space horizon- 
tally and two spaces vertically. That is, a knight on square 
(x, y) can move to any of the eight squares (x +2, y+ 1), 
(x +1, y +2), if these squares are on the chessboard, as il- 
lustrated here. 


A knight’s tour is a sequence of legal moves by a knight start- 
ing at some square and visiting each square exactly once. A 
knight’s tour is called reentrant if there is a legal move that 
takes the knight from the last square of the tour back to where 
the tour began. We can model knight’s tours using the graph 
that has a vertex for each square on the board, with an edge 
connecting two vertices if a knight can legally move between 
the squares represented by these vertices. 


56. Draw the graph that represents the legal moves of a knight 
on a3 x 3 chessboard. 


57. Draw the graph that represents the legal moves of a knight 
on a3 x 4 chessboard. 


58. a) Show that finding a knight’s tour on an m x n chess- 
board is equivalent to finding a Hamilton path on the 
graph representing the legal moves of a knight on that 
board. 


b) Show that finding a reentrant knight’s tour on an 
m x n chessboard is equivalent to finding a Hamil- 
ton circuit on the corresponding graph. 


*59, Show that there is a knight’s tour on a3 x 4 chessboard. 
*60. Show that there is no knight’s tour on a3 x 3 chessboard. 


*61. Show that there is no knight’s tour on a4 x 4 chessboard. 


Shortest-Path Problems 


62. 


63. 


* 64, 


65. 


* 66. 
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Show that the graph representing the legal moves of a 
knight on an m x n chessboard, whenever m and n are 
positive integers, is bipartite. 

Show that there is no reentrant knight’s tour on anm x n 
chessboard when m and n are both odd. [Hint: Use Exer- 
cises 55, 58b, and 62.] 

Show that there is a knight’s tour on an 8 x 8 chessboard. 
[Hint: You can construct a knight’s tour using a method 
invented by H. C. Warnsdorff in 1823: Start in any square, 
and then always move to a square connected to the fewest 
number of unused squares. Although this method may not 
always produce a knight’s tour, it often does. ] 


The parts of this exercise outline a proof of Ore’s theo- 
rem. Suppose that G is a simple graph with n vertices, 
n > 3, and deg(x) + deg(y) > n whenever x and y are 
nonadjacent vertices in G. Ore’s theorem states that under 
these conditions, G has a Hamilton circuit. 


a) Show that if G does not have a Hamilton circuit, then 
there exists another graph H with the same vertices 
as G, which can be constructed by adding edges to G 
such that the addition of a single edge would produce 
a Hamilton circuit in H. [Hint: Add as many edges 
as possible at each successive vertex of G without 
producing a Hamilton circuit.] 

b) Show that there is a Hamilton path in H. 

c) Let v1, v2,..., Vn be a Hamilton path in H. Show 
that deg(v,) + deg(v,) > n and that there are at most 
deg(v1) vertices not adjacent to v, (including v, it- 
self). 

d) Let S be the set of vertices preceding each vertex adja- 
cent to vj in the Hamilton path. Show that S contains 
deg(v1) vertices and v, ¢ S. 

e) Show that S contains a vertex vz, which is adjacent 
to vy, implying that there are edges connecting vj and 
Vez and vz and vy. 

f) Show that part (e) implies that vj, v2,..., ve—1, 
Vk, Vny Vn—1s +++» Ve+1, V1 iS a Hamilton circuit in G. 
Conclude from this contradiction that Ore’s theorem 
holds. 


Show that the worst case computational complexity of Al- 
gorithm | for finding Euler circuits in a connected graph 
with all vertices of even degree is O(m), where m is the 
number of edges of G. 


Introduction 


Many problems can be modeled using graphs with weights assigned to their edges. As an 
illustration, consider how an airline system can be modeled. We set up the basic graph model 
by representing cities by vertices and flights by edges. Problems involving distances can be 
modeled by assigning distances between cities to the edges. Problems involving flight time can 
be modeled by assigning flight times to edges. Problems involving fares can be modeled by 
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FIGURE 1 Weighted Graphs Modeling an Airline System. 


assigning fares to the edges. Figure | displays three different assignments of weights to the 
edges of a graph representing distances, flight times, and fares, respectively. 

Graphs that have a number assigned to each edge are called weighted graphs. Weighted 
graphs are used to model computer networks. Communications costs (such as the monthly cost 
of leasing a telephone line), the response times of the computers over these lines, or the distance 
between computers, can all be studied using weighted graphs. Figure 2 displays weighted graphs 
that represent three ways to assign weights to the edges of a graph of a computer network, 
corresponding to distance, response time, and cost. 

Several types of problems involving weighted graphs arise frequently. Determining a path 
of least length between two vertices in a network is one such problem. To be more specific, let 
the length of a path in a weighted graph be the sum of the weights of the edges of this path. (The 
reader should note that this use of the term length is different from the use of length to denote the 
number of edges in a path in a graph without weights.) The question is: What is a shortest path, 
that is, a path of least length, between two given vertices? For instance, in the airline system 
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FIGURE 2 Weighted Graphs Modeling a Computer Network. 


represented by the weighted graph shown in Figure 1, what is a shortest path in air distance 
between Boston and Los Angeles? What combinations of flights has the smallest total flight 
time (that is, total time in the air, not including time between flights) between Boston and Los 
Angeles? What is the cheapest fare between these two cities? In the computer network shown in 
Figure 2, what is a least expensive set of telephone lines needed to connect the computers in San 
Francisco with those in New York? Which set of telephone lines gives a fastest response time 
for communications between San Francisco and New York? Which set of lines has a shortest 
overall distance? 

Another important problem involving weighted graphs asks for a circuit of shortest total 
length that visits every vertex of a complete graph exactly once. This is the famous traveling 
salesperson problem, which asks for an order in which a salesperson should visit each of the 
cities on his route exactly once so that he travels the minimum total distance. We will discuss 
the traveling salesperson problem later in this section. 


A Shortest-Path Algorithm 


There are several different algorithms that find a shortest path between two vertices in a weighted 
graph. We will present a greedy algorithm discovered by the Dutch mathematician Edsger Di- 
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FIGURE 3 A Weighted Simple Graph. 


jkstra in 1959. The version we will describe solves this problem in undirected weighted graphs 
where all the weights are positive. It is easy to adapt it to solve shortest-path problems in directed 
graphs. 

Before giving a formal presentation of the algorithm, we will give an illustrative example. 


What is the length of a shortest path between a and z in the weighted graph shown in Figure 3? 


Solution: Although a shortest path is easily found by inspection, we will develop some ideas 
useful in understanding Dijkstra’s algorithm. We will solve this problem by finding the length 
of a shortest path from a to successive vertices, until z is reached. 

The only paths starting at a that contain no vertex other than a are formed by adding an 
edge that has a as one endpoint. These paths have only one edge. They are a, b of length 4 and 
a, d of length 2. It follows that d is the closest vertex to a, and the shortest path from a to d has 
length 2. 

We can find the second closest vertex by examining all paths that begin with the shortest 
path from a to a vertex in the set {a, d}, followed by an edge that has one endpoint in {a, d} and 
its other endpoint not in this set. There are two such paths to consider, a, d, e of length 7 and 
a, b of length 4. Hence, the second closest vertex to a is b and the shortest path from a to b has 
length 4. 

To find the third closest vertex to a, we need examine only the paths that begin with the 
shortest path from a to a vertex in the set {a, d, b}, followed by an edge that has one endpoint 
in the set {a, d, b} and its other endpoint not in this set. There are three such paths, a, b, c of 
length 7, a, b, e of length 7, and a, d, e of length 5. Because the shortest of these paths is a, d, e, 
the third closest vertex to a is e and the length of the shortest path from a to e is 5. 


EDSGER WYBE DIJKSTRA (1930-2002) Edsger Dijkstra, born in the Netherlands, began programming 
computers in the early 1950s while studying theoretical physics at the University of Leiden. In 1952, realizing that 
he was more interested in programming than in physics, he quickly completed the requirements for his physics 
degree and began his career as a programmer, even though programming was not a recognized profession. (In 
1957, the authorities in Amsterdam refused to accept “programming” as his profession on his marriage license. 
However, they did accept “theoretical physicist” when he changed his entry to this.) 

Dijkstra was one of the most forceful proponents of programming as a scientific discipline. He has made 
fundamental contributions to the areas of operating systems, including deadlock avoidance; programming lan- 
guages, including the notion of structured programming; and algorithms. In 1972 Dijkstra received the Turing 


Award from the Association for Computing Machinery, one of the most prestigious awards in computer science. Dijkstra became 
a Burroughs Research Fellow in 1973, and in 1984 he was appointed to a chair in Computer Science at the University of Texas, 


Austin. 
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To find the fourth closest vertex to a, we need examine only the paths that begin with the 
shortest path from a to a vertex in the set {a, d, b, e}, followed by an edge that has one endpoint 
in the set {a, d, b, e} and its other endpoint not in this set. There are two such paths, a, b, c of 
length 7 and a, d, e, z of length 6. Because the shorter of these paths is a, d, e, z, the fourth 
closest vertex to a is z and the length of the shortest path from a to z is 6. < 


Example | illustrates the general principles used in Dijkstra’s algorithm. Note that a shortest 
path from a to z could have been found by a brute force approach by examining the length of 
every path from a to z. However, this brute force approach is impractical for humans and even 
for computers for graphs with a large number of edges. 

We will now consider the general problem of finding the length of a shortest path between 
a and z in an undirected connected simple weighted graph. Dijkstra’s algorithm proceeds by 
finding the length of a shortest path from a to a first vertex, the length of a shortest path from 
a to a second vertex, and so on, until the length of a shortest path from a to z is found. As a 
side benefit, this algorithm is easily extended to find the length of the shortest path from a to all 
other vertices of the graph, and not just to z. 

The algorithm relies on a series of iterations. A distinguished set of vertices is constructed 
by adding one vertex at each iteration. A labeling procedure is carried out at each iteration. In 
this labeling procedure, a vertex w is labeled with the length of a shortest path from a to w that 
contains only vertices already in the distinguished set. The vertex added to the distinguished set 
is one with a minimal label among those vertices not already in the set. 

We now give the details of Dijkstra’s algorithm. It begins by labeling a with 0 and the 
other vertices with oo. We use the notation Lo(a) = 0 and Lo(v) = o for these labels before 
any iterations have taken place (the subscript 0 stands for the “Oth” iteration). These labels are 
the lengths of shortest paths from a to the vertices, where the paths contain only the vertex a. 
(Because no path from a to a vertex different from a exists, co is the length of a shortest path 
between a and this vertex.) 

Dijkstra’s algorithm proceeds by forming a distinguished set of vertices. Let S; denote this 
set after k iterations of the labeling procedure. We begin with So = Y. The set S; is formed 
from S,_; by adding a vertex u not in S;—; with the smallest label. 

Once u is added to S;, we update the labels of all vertices not in S;, so that L;(v), the 
label of the vertex v at the kth stage, is the length of a shortest path from a to v that contains 
vertices only in S,; (that is, vertices that were already in the distinguished set together with wu). 
Note that the way we choose the vertex u to add to S, at each step is an optimal choice at each 
step, making this a greedy algorithm. (We will prove shortly that this greedy algorithm always 
produces an optimal solution.) 

Let v be a vertex not in Sx. To update the label of v, note that L;(v) is the length of a shortest 
path from a to v containing only vertices in S;. The updating can be carried out efficiently when 
this observation is used: A shortest path from a to v containing only elements of S, is either a 
shortest path from a to v that contains only elements of S;_ (that is, the distinguished vertices 
not including w), or it is a shortest path from a to u at the (k — 1)st stage with the edge {u, v} 
added. In other words, 


Ly(a, v) = min{Ly_1(a, v), Le-1(a@, u) + wu, v)}, 


where w(u, v) is the length of the edge with u and v as endpoints. This procedure is iterated by 
successively adding vertices to the distinguished set until z is added. When z is added to the 
distinguished set, its label is the length of a shortest path from a to z. 

Dijkstra’s algorithm is given in Algorithm 1. Later we will give a proof that this algorithm 
is correct. Note that we can find the length of the shortest path from a to all other vertices of the 
graph if we continue this procedure until all vertices are added to the distinguished set. 
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EXAMPLE 2 


ALGORITHM 1 Dijkstra’s Algorithm. 


procedure Dijkstra(G: weighted connected simple graph, with 
all weights positive) 

{G has vertices a = vo, v],..-, Vn = z and lengths w(vj, v;) 
where w(v;, vj) = 00 if {v;, vj} is not an edge in G} 

fori := 1 ton 
L(v;) := co 

L(a) :=0 

S:=O 

{the labels are now initialized so that the label of a is 0 and all 
other labels are oo, and S is the empty set} 


while z ZS 
u := a vertex not in S with L(uw) minimal 
S:= SU {u} 


for all vertices v not in § 
if L(u) + wu, v) < L(v) then L(v) := L(u) + w(u, v) 
{this adds a vertex to § with minimal label and updates the 
labels of vertices not in S} 
return L(z) {L(z) = length of a shortest path from a to z} 


Example 2 illustrates how Dijkstra’s algorithm works. Afterward, we will show that this 
algorithm always produces the length of a shortest path between two vertices in a weighted 
graph. 


Use Dijkstra’s algorithm to find the length of a shortest path between the vertices a and z in the 
weighted graph displayed in Figure 4(a). 


Solution: The steps used by Dijkstra’s algorithm to find a shortest path between a and z are shown 
in Figure 4. At each iteration of the algorithm the vertices of the set S, are circled. A shortest 
path from a to each vertex containing only vertices in S; is indicated for each iteration. The 
algorithm terminates when z is circled. We find that a shortest path from a to z isa, c, b, d, e, Z, 
with length 13. < 


Remark: In performing Dijkstra’s algorithm it is sometimes more convenient to keep track of 
labels of vertices in each step using a table instead of redrawing the graph for each step. 


Next, we use an inductive argument to show that Dijkstra’s algorithm produces the length 
of a shortest path between two vertices a and z in an undirected connected weighted graph. Take 
as the inductive hypothesis the following assertion: At the kth iteration 


(i) the label of every vertex v in S is the length of a shortest path from a to this vertex, and 


(ii) the label of every vertex not in S is the length of a shortest path from a to this vertex that 
contains only (besides the vertex itself) vertices in S. 


When k = 0, before any iterations are carried out, S = 4, so the length of a shortest path from 
a to a vertex other than a is oo. Hence, the basis case is true. 
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FIGURE4 Using Dijkstra’s Algorithm to Find a Shortest Path from a to z. 
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THEOREM 1 


Assume that the inductive hypothesis holds for the kth iteration. Let v be the vertex added 
to S$ at the (k + 1)st iteration, so v is a vertex not in S at the end of the kth iteration with the 
smallest label (in the case of ties, any vertex with smallest label may be used). 

From the inductive hypothesis we see that the vertices in S before the (k + 1)st iteration 
are labeled with the length of a shortest path from a. Also, v must be labeled with the length of 
a shortest path to it from a. If this were not the case, at the end of the kth iteration there would 
be a path of length less than L;(v) containing a vertex not in S [because L;(v) is the length of a 
shortest path from a to v containing only vertices in S after the kth iteration]. Let u be the first 
vertex not in S in such a path. There is a path with length less than Lx (v) from a to u containing 
only vertices of S. This contradicts the choice of v. Hence, (i) holds at the end of the (k + 1)st 
iteration. 

Let u be a vertex not in S after k + 1 iterations. A shortest path from a to u containing only 
elements of S either contains v or it does not. If it does not contain v, then by the inductive 
hypothesis its length is L;(u). If it does contain v, then it must be made up of a path from a to v 
of shortest possible length containing elements of S other than v, followed by the edge from v 
to u. In this case, its length would be Lx(v) + w(v, u). This shows that (i7) is true, because 
Leyi(u) = min{Lx(u), Lev) + wQ, w)}. 

We now state the thereom that we have proved. 


Dijkstra’s algorithm finds the length of a shortest path between two vertices in a connected 
simple undirected weighted graph. 


714 


10 / Graphs 


THEOREM 2 


Links 


Saginaw 


Grand Rapids 


© Detroit 


Toledo 


FIGURE 5 The Graph Showing the Distances between Five Cities. 


We can now estimate the computational complexity of Dijkstra’s algorithm (in terms of 
additions and comparisons). The algorithm uses no more than n — | iterations where n is the 
number of vertices in the graph, because one vertex is added to the distinguished set at each 
iteration. We are done if we can estimate the number of operations used for each iteration. We 
can identify the vertex not in S; with the smallest label using no more than n — 1 comparisons. 
Then we use an addition and a comparison to update the label of each vertex not in Sx. It follows 
that no more than 2(n — 1) operations are used at each iteration, because there are no more 
than n — 1 labels to update at each iteration. Because we use no more than n — | iterations, each 
using no more than 2(n — 1) operations, we have Theorem 2. 


Dijkstra’s algorithm uses O(n”) operations (additions and comparisons) to find the length of 
a shortest path between two vertices in a connected simple undirected weighted graph with n 
vertices. 


The Traveling Salesperson Problem 


We now discuss an important problem involving weighted graphs. Consider the following prob- 
lem: A traveling salesperson wants to visit each of n cities exactly once and return to his starting 
point. For example, suppose that the salesperson wants to visit Detroit, Toledo, Saginaw, Grand 
Rapids, and Kalamazoo (see Figure 5). In which order should he visit these cities to travel the 
minimum total distance? To solve this problem we can assume the salesperson starts in Detroit 
(because this must be part of the circuit) and examine all possible ways for him to visit the other 
four cities and then return to Detroit (starting elsewhere will produce the same circuits). There 
are a total of 24 such circuits, but because we travel the same distance when we travel a circuit in 
reverse order, we need only consider 12 different circuits to find the minimum total distance he 
must travel. We list these 12 different circuits and the total distance traveled for each circuit. As 
can be seen from the list, the minimum total distance of 458 miles is traveled using the circuit 
Detroit-Toledo—Kalamazoo—Grand Rapids—Saginaw-Detroit (or its reverse). 


An 1832 handbook Der 
Handlungsreisende (The 
Traveling Salesman) 
mentions the traveling 
salesman problem, with 
sample tours through 
Germany and 
Switzerland. 


10.6 Shortest-Path Problems 715 


Total Distance (miles) 


Detroit-Toledo—Grand Rapids—Saginaw—Kalamazoo—Detroit 
Detroit-Toledo—Grand Rapids—Kalamazoo-Saginaw—Detroit 
Detroit-Toledo—Kalamazoo—Saginaw—Grand Rapids—Detroit 
Detroit-Toledo—Kalamazoo—Grand Rapids—Saginaw—Detroit 
Detroit-Toledo—Saginaw—Kalamazoo—Grand Rapids—Detroit 
Detroit-Toledo—Saginaw—Grand Rapids—Kalamazoo—Detroit 
Detroit-Saginaw—Toledo—Grand Rapids—Kalamazoo—Detroit 
Detroit-Saginaw—Toledo—Kalamazoo—Grand Rapids—Detroit 


Detroit-Saginaw—Kalamazoo—Toledo—Grand Rapids—Detroit 


Detroit-Saginaw—Grand Rapids—Toledo—Kalamazoo—Detroit 
Detroit-Grand Rapids—Saginaw—Toledo—Kalamazoo—Detroit 
Detroit-Grand Rapids—Toledo—Saginaw—Kalamazoo—Detroit 


We just described an instance of the traveling salesperson problem. The traveling sales- 
person problem asks for the circuit of minimum total weight in a weighted, complete, undirected 
graph that visits each vertex exactly once and returns to its starting point. This is equivalent to 
asking for a Hamilton circuit with minimum total weight in the complete graph, because each 
vertex is visited exactly once in the circuit. 

The most straightforward way to solve an instance of the traveling salesperson problem is 
to examine all possible Hamilton circuits and select one of minimum total length. How many 
circuits do we have to examine to solve the problem if there are n vertices in the graph? Once a 
starting point is chosen, there are (n — 1)! different Hamilton circuits to examine, because there 
are n — | choices for the second vertex, n — 2 choices for the third vertex, and so on. Because a 
Hamilton circuit can be traveled in reverse order, we need only examine (n — 1)!/2 circuits to find 
our answer. Note that (n — 1)!/2 grows extremely rapidly. Trying to solve a traveling salesperson 
problem in this way when there are only a few dozen vertices is impractical. For example, with 
25 vertices, a total of 24! /2 (approximately 3.1 x 1073) different Hamilton circuits would have 
to be considered. If it took just one nanosecond (10~° second) to examine each Hamilton circuit, 
a total of approximately ten million years would be required to find a minimum-length Hamilton 
circuit in this graph by exhaustive search techniques. 

Because the traveling salesperson problem has both practical and theoretical importance, 
a great deal of effort has been devoted to devising efficient algorithms that solve it. However, 
no algorithm with polynomial worst-case time complexity is known for solving this problem. 
Furthermore, if a polynomial worst-case time complexity algorithm were discovered for the 
traveling salesperson problem, many other difficult problems would also be solvable using 
polynomial worst-case time complexity algorithms (such as determining whether a proposition 
in n variables is a tautology, discussed in Chapter 1). This follows from the theory of NP- 
completeness. (For more information about this, consult [GaJo79].) 

A practical approach to the traveling salesperson problem when there are many vertices 
to visit is to use an approximation algorithm. These are algorithms that do not necessarily 
produce the exact solution to the problem but instead are guaranteed to produce a solution 
that is close to an exact solution. (Also, see the preamble to Exercise 46 in the Supplmentary 
Exercises of Chapter 3.) That is, they may produce a Hamilton circuit with total weight W’ such 
that W < W’ <cW, where W is the total length of an exact solution and c is a constant. For 
example, there is an algorithm with polynomial worst-case time complexity that works if the 
weighted graph satisfies the triangle inequality such that c = 3/2. For general weighted graphs 
for every positive real number k no algorithm is known that will always produce a solution at 
most k times a best solution. If such an algorithm existed, this would show that the class P would 
be the same as the class NP, perhaps the most famous open question about the complexity of 
algorithms (see Section 3.3). 
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In practice, algorithms have been developed that can solve traveling salesperson problems 
with as many as 1000 vertices within 2% of an exact solution using only a few minutes of 
computer time. For more information about the traveling salesperson problem, including his- 
tory, applications, and algorithms, see the chapter on this topic in Applications of Discrete 
Mathematics [MiRo91] also available on the website for this book. 


Exercises 


1. 


For each of these problems about a subway system, de- 
scribe a weighted graph model that can be used to solve 
the problem. 


a) What is the least amount of time required to travel 
between two stops? 


b) What is the minimum distance that can be traveled to 
reach a stop from another stop? 


c) What is the least fare required to travel between two 
stops if fares between stops are added to give the total 
fare? 


In Exercises 2—4 find the length of a shortest path between a 
and z in the given weighted graph. 


. Find a shortest path between a and z in each of the 


weighted graphs in Exercises 2-4. 


. Find the length of a shortest path between these pairs of 


vertices in the weighted graph in Exercise 3. 
a) aandd 


b) aand f 
c) cand f 
d) band z 


7. 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


Find shortest paths in the weighted graph in Exercise 3 
between the pairs of vertices in Exercise 6. 


. Find a shortest path (in mileage) between each of the 


following pairs of cities in the airline system shown in 
Figure 1. 


a) New York and Los Angeles 
b) Boston and San Francisco 
c) Miami and Denver 


d) Miami and Los Angeles 


. Find a combination of flights with the least total air time 


between the pairs of cities in Exercise 8, using the flight 
times shown in Figure 1. 


Find a least expensive combination of flights connecting 
the pairs of cities in Exercise 8, using the fares shown in 
Figure 1. 


Find a shortest route (in distance) between computer cen- 
ters in each of these pairs of cities in the communications 
network shown in Figure 2. 


a) Boston and Los Angeles 

b) New York and San Francisco 
c) Dallas and San Francisco 

d) Denver and New York 


Find a route with the shortest response time between the 
pairs of computer centers in Exercise 11 using the re- 
sponse times given in Figure 2. 


Find a least expensive route, in monthly lease charges, be- 
tween the pairs of computer centers in Exercise 11 using 
the lease charges given in Figure 2. 


Explain how to find a path with the least number of edges 
between two vertices in an undirected graph by consid- 
ering it as a shortest path problem in a weighted graph. 


Extend Dijkstra’s algorithm for finding the length of a 
shortest path between two vertices in a weighted simple 
connected graph so that the length of a shortest path be- 
tween the vertex a and every other vertex of the graph is 
found. 


Extend Dijkstra’s algorithm for finding the length of a 
shortest path between two vertices in a weighted simple 
connected graph so that a shortest path between these 
vertices is constructed. 


17. The weighted graphs in the figures here show some major 
roads in New Jersey. Part (a) shows the distances between 
cities on these roads; part (b) shows the tolls. 


Newark 


42 Woodbridge 
Sa 3 


(b) 


a) Finda shortest route in distance between Newark and 
Camden, and between Newark and Cape May, using 
these roads. 


b) Find a least expensive route in terms of total tolls us- 
ing the roads in the graph between the pairs of cities 
in part (a) of this exercise. 


18. Is ashortest path between two vertices in a weighted graph 
unique if the weights of edges are distinct? 


19. What are some applications where it is necessary to find 
the length of a longest simple path between two vertices 
in a weighted graph? 


20. What is the length of a longest simple path in the weighted 
graph in Figure 4 between a and z? Between c and z? 


Floyd’s algorithm, displayed as Algorithm 2, can be used to 
find the length of a shortest path between all pairs of vertices 
in a weighted connected simple graph. However, this algo- 
rithm cannot be used to construct shortest paths. (We assign 
an infinite weight to any pair of vertices not connected by an 
edge in the graph.) 
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21. Use Floyd’s algorithm to find the distance between all 
pairs of vertices in the weighted graph in Figure 4(a). 


* 22. Prove that Floyd’s algorithm determines the shortest dis- 
tance between all pairs of vertices in a weighted simple 
graph. 


* 23. Give a big-O estimate of the number of operations (com- 
parisons and additions) used by Floyd’s algorithm to de- 
termine the shortest distance between every pair of ver- 
tices in a weighted simple graph with n vertices. 


* 24, Show that Dijkstra’s algorithm may not work if edges can 
have negative weights. 


ALGORITHM 2 Floyd’s Algorithm. 


procedure Floyd(G: weighted simple graph) 
{G has vertices v], v2,..., V, and weights w(vj, v;) 
with w(v;, vj) = oo if {v;, vj} is not an edge} 
fori := 1 ton 
for j := 1 ton 
dvi, Vj) = wi, vj) 
fori := 1 ton 
for j := 1 ton 
for k := 1 ton 
if d(vj, vi) + di, Vk) < d(Vj, Ve) 
then d(vj;, vg) := d(vj, vi) + di, ve) 
return [d(i, v)] {d(vj, v;) is the length of a shortest 
path between v; and v; forl <i <n,1 <j <n} 


25. Solve the traveling salesperson problem for this graph 
by finding the total weight of all Hamilton circuits and 
determining a circuit with minimum total weight. 


26. Solve the traveling salesperson problem for this graph 
by finding the total weight of all Hamilton circuits and 
determining a circuit with minimum total weight. 
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27. Find a route with the least total airfare that visits each of 
the cities in this graph, where the weight on an edge is the 
least price available for a flight between the two cities. 


329 Detroit 

: $ 

San S a9) 

Francisco ey Gr _ $279 New 
York 

Ww) 
$69 Y 
Los Angeles Denver 


28. Find a route with the least total airfare that visits each of 
the cities in this graph, where the weight on an edge is the 
least price available for a flight between the two cities. 


Seattle $409 


Phoenix $309 


New Orleans 
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29. Construct a weighted undirected graph such that the total 
weight of a circuit that visits every vertex at least once 
is minimized for a circuit that visits some vertices more 
than once. [Hint: There are examples with three vertices. ] 


30. Show that the problem of finding a circuit of minimum 
total weight that visits every vertex of a weighted graph 
at least once can be reduced to the problem of finding a 
circuit of minimum total weight that visits each vertex 
of a weighted graph exactly once. Do so by constructing 
anew weighted graph with the same vertices and edges 
as the original graph but whose weight of the edge con- 
necting the vertices u and v is equal to the minimum total 
weight of a path from u to v in the original graph. 


*31. The longest path problem in a weighted directed graph 
with no simple circuits asks for a path in this graph such 
that the sum of its edge weights is a maximum. De- 
vise an algorithm for solving the longest path problem. 
(Hint: First find a topological ordering of the vertices of 
the graph.] 


Introduction 


Consider the problem of joining three houses to each of three separate utilities, as shown in 
4 Figure 1. Is it possible to join these houses and utilities so that none of the connections cross? 
Links Qi This problem can be modeled using the complete bipartite graph K3,3. The original question 
can be rephrased as: Can K3.3 be drawn in the plane so that no two of its edges cross? 
In this section we will study the question of whether a graph can be drawn in the plane 
without edges crossing. In particular, we will answer the houses-and-utilities problem. 
There are always many ways to represent a graph. When is it possible to find at least one 
way to represent this graph in a plane without any edges crossing? 


FIGURE 1 Three Houses and Three Utilities. 


x 


FIGURE2 The 
Graph K4. 


DEFINITION 1 


EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


V4 
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N le 


FIGURE3 K4 Drawn FIGURE4 The FIGURES A Planar 
with No Crossings. Graph Q3. Representation of Q3. 


A graph is called planar if it can be drawn in the plane without any edges crossing (where 
a crossing of edges is the intersection of the lines or arcs representing them at a point other 
than their common endpoint). Such a drawing is called a planar representation of the graph. 


A graph may be planar even if it is usually drawn with crossings, because it may be possible 
to draw it in a different way without crossings. 


Is K4 (shown in Figure 2 with two edges crossing) planar? 


Solution: K4 is planar because it can be drawn without crossings, as shown in Figure 3. < 


Is Q3, shown in Figure 4, planar? 


Solution: Q3 1s planar, because it can be drawn without any edges crossing, as shown in 
Figure 5. < 


We can show that a graph is planar by displaying a planar representation. It is harder to 
show that a graph is nonplanar. We will give an example to show how this can be done in an ad 
hoc fashion. Later we will develop some general results that can be used to do this. 


Is K33, shown in Figure 6, planar? 


Solution: Any attempt to draw K3,3 in the plane with no edges crossing is doomed. We now 
show why. In any planar representation of K3,3, the vertices v; and vz must be connected to both 
v4 and vs. These four edges form a closed curve that splits the plane into two regions, R; and 
Ro, as shown in Figure 7(a). The vertex v3 is in either R; or R2. When v3 is in R2, the inside 
of the closed curve, the edges between v3 and vq and between v3 and vs separate R2 into two 
subregions, R2; and R22, as shown in Figure 7(b). 


V5 % (a) (b) 


FIGURE 6 The Graph K3,3. FIGURE7 Showing that K3,3 Is Nonplanar. 


720 


10 / Graphs 


THEOREM 1 


? 


Next, note that there is no way to place the final vertex v6 without forcing a crossing. For 
if v6 is in Rj, then the edge between ve and v3 cannot be drawn without a crossing. If v6 is in 
Ro, then the edge between v2 and ve cannot be drawn without a crossing. If vg is in R22, then 
the edge between v, and v6 cannot be drawn without a crossing. 

A similar argument can be used when v3 is in Ry. The completion of this argument is left 
for the reader (see Exercise 10). It follows that K3 3 is not planar. 


Example 3 solves the utilities-and-houses problem that was described at the beginning of 
this section. The three houses and three utilities cannot be connected in the plane without a 
crossing. A similar argument can be used to show that Ks is nonplanar. (See Exercise 11.) 


APPLICATIONS OF PLANAR GRAPHS | Planarity of graphs plays an important role in the 
design of electronic circuits. We can model a circuit with a graph by representing components 
of the circuit by vertices and connections between them by edges. We can print a circuit on a 
single board with no connections crossing if the graph representing the circuit is planar. When 
this graph is not planar, we must turn to more expensive options. For example, we can partition 
the vertices in the graph representing the circuit into planar subgraphs. We then construct the 
circuit using multiple layers. (See the preamble to Exercise 30 to learn about the thickness of a 
graph.) We can construct the circuit using insulated wires whenever connections cross. In this 
case, drawing the graph with the fewest possible crossings is important. (See the preamble to 
Exercise 26 to learn about the crossing number of a graph.) 

The planarity of graphs is also useful in the design of road networks. Suppose we want 
to connect a group of cities by roads. We can model a road network connecting these cities 
using a simple graph with vertices representing the cities and edges representing the highways 
connecting them. We can built this road network without using underpasses or overpasses if the 
resulting graph is planar. 


Euler’s Formula 


A planar representation of a graph splits the plane into regions, including an unbounded region. 
For instance, the planar representation of the graph shown in Figure 8 splits the plane into six 
regions. These are labeled in the figure. Euler showed that all planar representations of a graph 
split the plane into the same number of regions. He accomplished this by finding a relationship 
among the number of regions, the number of vertices, and the number of edges of a planar graph. 


EULER’S FORMULA _ Let G be a connected planar simple graph with e edges and v 
vertices. Let r be the number of regions in a planar representation of G. Thenr = e —v+2. 


Proof: First, we specify a planar representation of G. We will prove the theorem by constructing 
a sequence of subgraphs G, G2,..., Ge = G, successively adding an edge at each stage. This 
is done using the following inductive definition. Arbitrarily pick one edge of G to obtain G,. 
Obtain G,, from G,,_; by arbitrarily adding an edge that is incident with a vertex already in Gy_1, 


FIGURE 8 The Regions of the Planar Representation of a Graph. 


ay bd 


FIGURE9 The 
Basis Case of the 
Proof of Euler’s 
Formula. 


EXAMPLE 4 
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adding the other vertex incident with this edge if it is not already in G,_1. This construction 
is possible because G is connected. G is obtained after e edges are added. Let ry, e,, and vp, 
represent the number of regions, edges, and vertices of the planar representation of G,, induced 
by the planar representation of G, respectively. 

The proof will now proceed by induction. The relationship 71 = e; — v; + 2 is true for Gy, 
because ej = 1, vj = 2, andr; = 1. This is shown in Figure 9. 

Now assume that rp, = eg, — ve + 2. Let {ag+1, bg41} be the edge that is added to Gx to 
obtain Gx+,. There are two possibilities to consider. In the first case, both ag4 1 and by+ 1 are 
already in Gz. These two vertices must be on the boundary of a common region R, or else 
it would be impossible to add the edge {az+1, by+1} to Gg without two edges crossing (and 
Gx+1 is planar). The addition of this new edge splits R into two regions. Consequently, in this 
case, rea. = re +1, eke = ex +1, and vg4) = vg. Thus, each side of the formula relating the 
number of regions, edges, and vertices increases by exactly one, so this formula is still true. In 
other words, p41 = €k+1 — Ve41 + 2. This case is illustrated in Figure 10(a). 

In the second case, one of the two vertices of the new edge is not already in Gx. Suppose 
that ag4 1 is in Gx but that by+) is not. Adding this new edge does not produce any new regions, 
because bg4 ; must be in a region that has ag; on its boundary. Consequently, rp4.1 = rz. 
Moreover, ex41 = ex +1 and vgs) = ve + 1. Each side of the formula relating the number 
of regions, edges, and vertices remains the same, so the formula is still true. In other words, 
rk+l = Ck+1 — Ve+1 + 2. This case is illustrated in Figure 10(b). 

We have completed the induction argument. Hence, r;, = e, — v, + 2 for alln. Because the 
original graph is the graph G;, obtained after e edges have been added, the theorem is true. <1 


Euler’s formula is illustrated in Example 4. 


Suppose that a connected planar simple graph has 20 vertices, each of degree 3. Into how many 
regions does a representation of this planar graph split the plane? 


Solution: This graph has 20 vertices, each of degree 3, so v = 20. Because the sum of the degrees 


of the vertices, 3v = 3 - 20 = 60, is equal to twice the number of edges, 2e, we have 2e = 60, 
or e = 30. Consequently, from Euler’s formula, the number of regions is 


r=e-—v+2=30-—-204+2=12. 4 


Euler’s formula can be used to establish some inequalities that must be satisfied by planar 
graphs. One such inequality is given in Corollary 1. 


Gn+1 


b 


n+1 
(a) (b) 


FIGURE 10 Adding an Edge to G,, to Produce G,41. 
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COROLLARY 2 


i 


FIGURE 11 The Degrees of Regions. 


If G is a connected planar simple graph with e edges and v vertices, where v > 3, then 
eS 39 —, 


Before we prove Corollary | we will use it to prove the following useful result. 
If G is a connected planar simple graph, then G has a vertex of degree not exceeding five. 


Proof: If G has one or two vertices, the result is true. If G has at least three vertices, by Corollary | 
we know that e < 3v — 6, so 2e < 6v — 12. If the degree of every vertex were at least six, then 
because 2e = )°,,-y deg(v) (by the handshaking theorem), we would have 2e > 6v. But this 
contradicts the inequality 2e < 6v — 12. It follows that there must be a vertex with degree no 
greater than five. | 


The proof of Corollary | is based on the concept of the degree of a region, which is defined 
to be the number of edges on the boundary of this region. When an edge occurs twice on the 
boundary (so that it is traced out twice when the boundary is traced out), it contributes two to 
the degree. We denote the degree of a region R by deg(R). The degrees of the regions of the 
graph shown in Figure 11 are displayed in the figure. 

The proof of Corollary | can now be given. 


Proof: A connected planar simple graph drawn in the plane divides the plane into regions, 
say r of them. The degree of each region is at least three. (Because the graphs discussed here 
are simple graphs, no multiple edges that could produce regions of degree two, or loops that 
could produce regions of degree one, are permitted.) In particular, note that the degree of the 
unbounded region is at least three because there are at least three vertices in the graph. 

Note that the sum of the degrees of the regions is exactly twice the number of edges in 
the graph, because each edge occurs on the boundary of a region exactly twice (either in two 
different regions, or twice in the same region). Because each region has degree greater than or 
equal to three, it follows that 


2e = > deg(R) > 3r. 


all regions R 
Hence, 


(2/3)e =r. 
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Links 


10.7 Planar Graphs 723 


G, 


FIGURE 12 Homeomorphic Graphs. 


Using r = e — v+ 2 (Euler’s formula), we obtain 
e—v4+2< (2/3)e. 
It follows that e/3 < v — 2. This shows that e < 3v — 6. qd 


This corollary can be used to demonstrate that Ks is nonplanar. 


Show that Ks is nonplanar using Corollary 1. 


Solution: The graph Ks has five vertices and 10 edges. However, the inequality e < 3v — 6 is 
not satisfied for this graph because e = 10 and 3v — 6 = 9. Therefore, Ks is not planar. < 


It was previously shown that K3,3 is not planar. Note, however, that this graph has six vertices 
and nine edges. This means that the inequality e = 9 < 12 = 3 - 6 — 6is satisfied. Consequently, 
the fact that the inequality e < 3v — 6is satisfied does not imply that a graph is planar. However, 
the following corollary of Theorem | can be used to show that K3,3 is nonplanar. 


If a connected planar simple graph has e edges and v vertices with v > 3 and no circuits of 
length three, then e < 2v — 4. 


The proof of Corollary 3 is similar to that of Corollary 1, except that in this case the fact that 
there are no circuits of length three implies that the degree of a region must be at least four. The 
details of this proof are left for the reader (see Exercise 15). 


Use Corollary 3 to show that K3 3 is nonplanar. 


Solution: Because K3 3 has no circuits of length three (this is easy to see because it is bipartite), 
Corollary 3 can be used. K3,3 has six vertices and nine edges. Because e = 9 and 2v — 4 = 8, 
Corollary 3 shows that K3.3 is nonplanar. < 


KAZIMIERZ KURATOWSKI (1896-1980) Kazimierz Kuratowski, the son of a famous Warsaw lawyer, 
attended secondary school in Warsaw. He studied in Glasgow, Scotland, from 1913 to 1914 but could not return 
there after the outbreak of World War I. In 1915 he entered Warsaw University, where he was active in the Polish 
patriotic student movement. He published his first paper in 1919 and received his Ph.D. in 1921. He was an active 
member of the group known as the Warsaw School of Mathematics, working in the areas of the foundations 
of set theory and topology. He was appointed associate professor at the Lw6w Polytechnical University, where 
he stayed for seven years, collaborating with the important Polish mathematicians Banach and Ulam. In 1930, 
while at Lw6w, Kuratowski completed his work characterizing planar graphs. 

In 1934 he returned to Warsaw University as a full professor. Until the start of World War II, he was active 


in research and teaching. During the war, because of the persecution of educated Poles, Kuratowski went into hiding under an 
assumed name and taught at the clandestine Warsaw University. After the war he helped revive Polish mathematics, serving as 
director of the Polish National Mathematics Institute. He wrote over 180 papers and three widely used textbooks. 
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FIGURE 13 The Undirected Graph G, a Subgraph H Homeomorphic to Ks, and Ks. 


Kuratowski’s Theorem 


We have seen that K3 3 and Ks are not planar. Clearly, a graph is not planar if it contains either 
of these two graphs as a subgraph. Surprisingly, all nonplanar graphs must contain a subgraph 
that can be obtained from K3 3 or Ks using certain permitted operations. 

If a graph is planar, so will be any graph obtained by removing an edge {u, v} and adding a 
new vertex w together with edges {u, w} and {w, v}. Such an operation is called an elementary 
subdivision. The graphs G; = (V;, E,) and Gz = (V2, F2) are called homeomorphic if they 
can be obtained from the same graph by a sequence of elementary subdivisions. 


Show that the graphs G;, G2, and G3 displayed in Figure 12 are all homeomorphic. 


Solution: These three graphs are homeomorphic because all three can be obtained from G, by 
elementary subdivisions. G; can be obtained from itself by an empty sequence of elementary 
subdivisions. To obtain G2 from Gy, we can use this sequence of elementary subdivisions: 
(i) remove the edge {a, c}, add the vertex f, and add the edges {a, f} and { f, c}; (i) remove 
the edge {b, c}, add the vertex g, and add the edges {b, g} and {g, c}; and (iii ) remove the edge 
{b, g}, add the vertex h, and add the edges {g, h} and {b, h}. We leave it to the reader to determine 
the sequence of elementary subdivisions needed to obtain G3 from G}. < 


The Polish mathematician Kazimierz Kuratowski established Theorem 2 in 1930, which 
characterizes planar graphs using the concept of graph homeomorphism. 


A graph is nonplanar if and only if it contains a subgraph homeomorphic to K3.3 or Ks. 


It is clear that a graph containing a subgraph homeomorphic to K3 3 or Ks is nonplanar. However, 
the proof of the converse, namely that every nonplanar graph contains a subgraph homeomorphic 
to K33 or Ks, is complicated and will not be given here. Examples 8 and 9 illustrate how 
Kuratowski’s theorem is used. 


Determine whether the graph G shown in Figure 13 is planar. 


Solution: G has a subgraph H homeomorphic to K5. H is obtained by deleting h, j, and k and 
all edges incident with these vertices. H is homeomorphic to Ks because it can be obtained 
from K5 (with vertices a, b, c, g, andi) by a sequence of elementary subdivisions, adding the 
vertices d, e, and f. (The reader should construct such a sequence of elementary subdivisions.) 
Hence, G is nonplanar. 
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(a) (b) H (c) K33 


FIGURE 14 (a) The Petersen Graph, (b) a Subgraph H Homeomorphic to K3,3, and (c) K3,3. 


EXAMPLE 9 Is the Petersen graph, shown in Figure 14(a), planar? (The Danish mathematician Julius Petersen 
studied this graph in 1891; it is often used to illustrate various theoretical properties of graphs.) 


Solution: The subgraph H of the Petersen graph obtained by deleting b and the three edges 
that have b as an endpoint, shown in Figure 14(b), is homeomorphic to K3,3, with vertex sets 
{f, d, j} and {e, i, h}, because it can be obtained by a sequence of elementary subdivisions, 
deleting {d, h} and adding {c, h} and {c, d}, deleting {e, f} and adding {a, e} and {a, f}, and 
deleting {i, 7} and adding {g, 7} and {g, 7}. Hence, the Petersen graph is not planar. < 


Exercises 
1. Can five houses be connected to two utilities without con- 7. a b 8. a b 
nections crossing? 
In Exercises 2-4 draw the given planar graph without any h c 
crossings. f c 
g d 
2. 3. 
e d f e 
9. 


4. 


10. Complete the argument in Example 3. 


11. Show that Ks is nonplanar using an argument similar to 


In Exercises 5—9 determine whether the given graph is planar. that een in Ea amplee 


If so, draw it so that no edges cross. 12. Suppose that a connected planar graph has eight vertices, 
each of degree three. Into how many regions is the plane 
divided by a planar representation of this graph? 


a z “ ° a 13. Suppose that a connected planar graph has six vertices, 
each of degree four. Into how many regions is the plane 
b c divided by a planar representation of this graph? 
14. Suppose that a connected planar graph has 30 edges. Ifa 
é . . . - planar representation of this graph divides the plane into 
rs 


20 regions, how many vertices does this graph have? 
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15. 
16. 
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18. 


19. 
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Prove Corollary 3. 


Suppose that a connected bipartite planar simple graph 
has e edges and v vertices. Show that e < 2v — 4ifv > 3. 


Suppose that a connected planar simple graph with e 
edges and v vertices contains no simple circuits of length 
4 or less. Show that e < (5/3)v — (10/3) if v > 4. 


Suppose that a planar graph has k connected components, 
e edges, and v vertices. Also suppose that the plane is 
divided into 7 regions by a planar representation of the 
graph. Find a formula for r in terms of e, v, and k. 


Which of these nonplanar graphs have the property that 
the removal of any vertex and all edges incident with that 
vertex produces a planar graph? 


a) Ks b) Ke c) K33 d) K34 


In Exercises 20-22 determine whether the given graph is 
homeomorphic to K3 3. 


20. 


21. 


22. 


a b c 
&§ <p 
d 


In Exercises 23—25 use Kuratowski’s theorem to determine 
whether the given graph is planar. 


23. 


a b c d 


25. 


~~ The crossing number of a simple graph is the minimum num- 
ber of crossings that can occur when this graph is drawn in the 
plane where no three arcs representing edges are permitted to 
cross at the same point. 


26. 
#97. 


* 28. 
** 29, 


Show that K3 3 has | as its crossing number. 

Find the crossing numbers of each of these nonplanar 
graphs. 
a) Ks 
d) K34 


Find the crossing number of the Petersen graph. 

Show that if m and n are even positive integers, the cross- 
ing number of K7,,, 1s less than or equal to mn(m — 2) 
(n — 2)/16. [Hint: Place m vertices along the x-axis so 
that they are equally spaced and symmetric about the ori- 
gin and place n vertices along the y-axis so that they are 
equally spaced and symmetric about the origin. Now con- 
nect each of the m vertices on the x-axis to each of the 
vertices on the y-axis and count the crossings. ] 


b) Ko 
e) K44 


Cc) Ky 
f) K55 


The thickness of a simple graph G is the smallest number of 
planar subgraphs of G that have G as their union. 


30. 
*31. 
32. 


* 33. 


34. 


35. 


* 36. 


* 37. 


Show that K3 3 has 2 as its thickness. 

Find the thickness of the graphs in Exercise 27. 

Show that if G is aconnected simple graph with v vertices 
and e edges, where v > 3, then the thickness of G is at 
least [e/(3v — 6)]. 

Use Exercise 32 to show that the thickness of Ky, is at 
least |(n + 7)/6] whenever n is a positive integer. 

Show that if G is aconnected simple graph with v vertices 
and e edges, where v > 3, and no circuits of length three, 
then the thickness of G is at least [e/(2v — 4)]. 

Use Exercise 34 to show that the thickness of Kim,n, where 
m and n are not both 1, is at least [mn/(2m + 2n — 4)] 
whenever m and n are positive integers. 

Draw Ks on the surface of a torus (a doughnut-shaped 
solid) so that no edges cross. 

Draw K3,3 on the surface of a torus so that no edges cross. 
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FIGURE 1 Two Maps. 


Graph Coloring 


Links 


DEFINITION 1 


Introduction 


Problems related to the coloring of maps of regions, such as maps of parts of the world, have 
generated many results in graph theory. When a map” is colored, two regions with a common 
border are customarily assigned different colors. One way to ensure that two adjacent regions 
never have the same color is to use a different color for each region. However, this is inefficient, 
and on maps with many regions it would be hard to distinguish similar colors. Instead, a small 
number of colors should be used whenever possible. Consider the problem of determining the 
least number of colors that can be used to color a map so that adjacent regions never have the 
same color. For instance, for the map shown on the left in Figure 1, four colors suffice, but three 
colors are not enough. (The reader should check this.) In the map on the right in Figure 1, three 
colors are sufficient (but two are not). 

Each map in the plane can be represented by a graph. To set up this correspondence, 
each region of the map is represented by a vertex. Edges connect two vertices if the regions 
represented by these vertices have a common border. Two regions that touch at only one point 
are not considered adjacent. The resulting graph is called the dual graph of the map. By the way 
in which dual graphs of maps are constructed, it is clear that any map in the plane has a planar 
dual graph. Figure 2 displays the dual graphs that correspond to the maps shown in Figure 1. 

The problem of coloring the regions of a map is equivalent to the problem of coloring the 
vertices of the dual graph so that no two adjacent vertices in this graph have the same color. We 
now define a graph coloring. 


A coloring of a simple graph is the assignment of a color to each vertex of the graph so that 
no two adjacent vertices are assigned the same color. 


E 


FIGURE 2. Dual Graphs of the Maps in Figure 1. 


*We will assume that all regions in a map are connected. This eliminates any problems presented by such geographical entities 
as Michigan. 
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Links 


A graph can be colored by assigning a different color to each of its vertices. However, for most 
graphs a coloring can be found that uses fewer colors than the number of vertices in the graph. 
What is the least number of colors necessary? 


The chromatic number of a graph is the least number of colors needed for a coloring of 
this graph. The chromatic number of a graph G is denoted by x(G). (Here x is the Greek 
letter chi.) 


Note that asking for the chromatic number of a planar graph is the same as asking for the 
minimum number of colors required to color a planar map so that no two adjacent regions are 
assigned the same color. This question has been studied for more than 100 years. The answer is 
provided by one of the most famous theorems in mathematics. 


THE FOUR COLOR THEOREM The chromatic number of a planar graph is no greater 
than four. 


The four color theorem was originally posed as a conjecture in the 1850s. It was finally 
proved by the American mathematicians Kenneth Appel and Wolfgang Haken in 1976. Prior to 
1976, many incorrect proofs were published, often with hard-to-find errors. In addition, many 
futile attempts were made to construct counterexamples by drawing maps that require more than 
four colors. (Proving the five color theorem is not that difficult; see Exercise 36.) 

Perhaps the most notorious fallacious proof in all of mathematics is the incorrect proof of 
the four color theorem published in 1879 by a London barrister and amateur mathematician, 
Alfred Kempe. Mathematicians accepted his proof as correct until 1890, when Percy Heawood 
found an error that made Kempe’s argument incomplete. However, Kempe’s line of reasoning 
turned out to be the basis of the successful proof given by Appel and Haken. Their proof relies 
on a careful case-by-case analysis carried out by computer. They showed that if the four color 
theorem were false, there would have to be a counterexample of one of approximately 2000 
different types, and they then showed that none of these types exists. They used over 1000 
hours of computer time in their proof. This proof generated a large amount of controversy, 
because computers played such an important role in it. For example, could there be an error in a 
computer program that led to incorrect results? Was their argument really a proof if it depended 
on what could be unreliable computer output? Since their proof appeared, simpler proofs that 
rely on checking fewer types of possible counterexamples have been found and a proof using 
an automated proof system has been created. However, no proof not relying on a computer has 
yet been found. 

Note that the four color theorem applies only to planar graphs. Nonplanar graphs can have 
arbitrarily large chromatic numbers, as will be shown in Example 2. 

Two things are required to show that the chromatic number of a graph is k. First, we must 
show that the graph can be colored with k colors. This can be done by constructing such a 
coloring. Second, we must show that the graph cannot be colored using fewer than k colors. 
Examples 1-4 illustrate how chromatic numbers can be found. 


ALFRED BRAY KEMPE (1849-1922) Kempe was a barrister and a leading authority on ecclesiastical law. 
However, having studied mathematics at Cambridge University, he retained his interest in it, and later in life 


| he devoted considerable time to mathematical research. Kempe made contributions to kinematics, the branch 


of mathematics dealing with motion, and to mathematical logic. However, Kempe is best remembered for his 


| fallacious proof of the four color theorem. 
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FIGURE 3 The Simple Graphs G and H. 


What are the chromatic numbers of the graphs G and H shown in Figure 3? 


Solution: The chromatic number of G is at least three, because the vertices a, b, and c must 
be assigned different colors. To see if G can be colored with three colors, assign red to a, blue 
to b, and green to c. Then, d can (and must) be colored red because it is adjacent to b and c. 
Furthermore, e can (and must) be colored green because it is adjacent only to vertices colored 
red and blue, and f can (and must) be colored blue because it is adjacent only to vertices colored 
red and green. Finally, g can (and must) be colored red because it is adjacent only to vertices 
colored blue and green. This produces a coloring of G using exactly three colors. Figure 4 
displays such a coloring. 

The graph H is made up of the graph G with an edge connecting a and g. Any attempt to 
color H using three colors must follow the same reasoning as that used to color G, except at the 
last stage, when all vertices other than g have been colored. Then, because g is adjacent (in H) 
to vertices colored red, blue, and green, a fourth color, say brown, needs to be used. Hence, H 
has a chromatic number equal to 4. A coloring of H is shown in Figure 4. < 


b Blue e Green b Blue Green e 
g Red a g 
Red rown 
c ZS f Blue c aS f Blue 


FIGURE 4 _ Colorings of the Graphs G and H. 


HISTORICAL NOTE In 1852, an ex-student of Augustus De Morgan, Francis Guthrie, noticed that the 
counties in England could be colored using four colors so that no adjacent counties were assigned the same 
color. On this evidence, he conjectured that the four color theorem was true. Francis told his brother Frederick, 
at that time a student of De Morgan, about this problem. Frederick in turn asked his teacher De Morgan about 
his brother’s conjecture. De Morgan was extremely interested in this problem and publicized it throughout the 
mathematical community. In fact, the first written reference to the conjecture can be found in a letter from De 
Morgan to Sir William Rowan Hamilton. Although De Morgan thought Hamilton would be interested in this 
problem, Hamilton apparently was not interested in it, because it had nothing to do with quaternions. 


HISTORICAL NOTE Although a simpler proof of the four color theorem was found by Robertson, Sanders, 
Seymour, and Thomas in 1996, reducing the computational part of the proof to examining 633 configurations, 
no proof that does not rely on extensive computation has yet been found. 
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a Red b Blue 
a Red b Red c Red 
Brown e c Green PER 
d Yellow d Blue e Blue f Blue g Blue 
FIGURE 5 A Coloring of Ks. FIGURE 6 A Coloring of K3,4. 


What is the chromatic number of K,,? 


Solution: A coloring of Ky, can be constructed using n colors by assigning a different color 
to each vertex. Is there a coloring using fewer colors? The answer is no. No two vertices can 
be assigned the same color, because every two vertices of this graph are adjacent. Hence, the 
chromatic number of K,, is n. That is, x (Ky) =n. (Recall that K, is not planar when n > 5, 
so this result does not contradict the four color theorem.) A coloring of Ks using five colors is 
shown in Figure 5. < 


What is the chromatic number of the complete bipartite graph K;,,,, where m and n are positive 
integers? 


Solution: The number of colors needed may seem to depend on m and n. However, as Theorem 4 
in Section 10.2 tells us, only two colors are needed, because K;,,, is a bipartite graph. Hence, 
xX (Kin.n) = 2. This means that we can color the set of m vertices with one color and the set of 
n vertices with a second color. Because edges connect only a vertex from the set of m vertices 
and a vertex from the set of n vertices, no two adjacent vertices have the same color. A coloring 
of K3 4 with two colors is displayed in Figure 6. < 


What is the chromatic number of the graph C,,, where n > 3? (Recall that C,, is the cycle with 
n vertices.) 


Solution: We will first consider some individual cases. To begin, let n = 6. Pick a vertex and 
color it red. Proceed clockwise in the planar depiction of Ce shown in Figure 7. It is necessary to 
assign a second color, say blue, to the next vertex reached. Continue in the clockwise direction; 
the third vertex can be colored red, the fourth vertex blue, and the fifth vertex red. Finally, the 
sixth vertex, which is adjacent to the first, can be colored blue. Hence, the chromatic number of 
C¢ is 2. Figure 7 displays the coloring constructed here. 


FIGURE 7 Colorings of Cs and C6. 


Links 


EXAMPLE 5 
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Next, let n = 5 and consider Cs. Pick a vertex and color it red. Proceeding clockwise, it 
is necessary to assign a second color, say blue, to the next vertex reached. Continuing in the 
clockwise direction, the third vertex can be colored red, and the fourth vertex can be colored blue. 
The fifth vertex cannot be colored either red or blue, because it is adjacent to the fourth vertex 
and the first vertex. Consequently, a third color is required for this vertex. Note that we would 
have also needed three colors if we had colored vertices in the counterclockwise direction. Thus, 
the chromatic number of Cs is 3. A coloring of C5 using three colors is displayed in Figure 7. 

In general, two colors are needed to color C, when n is even. To construct such a coloring, 
simply pick a vertex and color it red. Proceed around the graph in a clockwise direction (using 
a planar representation of the graph) coloring the second vertex blue, the third vertex red, and 
so on. The nth vertex can be colored blue, because the two vertices adjacent to it, namely 
the(n — 1)st and the first vertices, are both colored red. 

When n is odd and n > 1, the chromatic number of C,, is 3. To see this, pick an initial 
vertex. To use only two colors, it is necessary to alternate colors as the graph is traversed in 
a clockwise direction. However, the nth vertex reached is adjacent to two vertices of different 
colors, namely, the first and (n — 1)st. Hence, a third color must be used. 

We have shown that x (C,,) = 2 if n is an even positive integer with n > 4 and x(C,) = 3 
if n is an odd positive integer with n > 3. < 


The best algorithms known for finding the chromatic number of a graph have exponential 
worst-case time complexity (in the number of vertices of the graph). Even the problem of finding 
an approximation to the chromatic number of a graph is difficult. It has been shown that if there 
were an algorithm with polynomial worst-case time complexity that could approximate the 
chromatic number of a graph up to a factor of 2 (that is, construct a bound that was no more 
than double the chromatic number of the graph), then an algorithm with polynomial worst-case 
time complexity for finding the chromatic number of the graph would also exist. 


Applications of Graph Colorings 


Graph coloring has a variety of applications to problems involving scheduling and assignments. 
(Note that because no efficient algorithm is known for graph coloring, this does not lead to 
efficient algorithms for scheduling and assignments.) Examples of such applications will be 
given here. The first application deals with the scheduling of final exams. 


Scheduling Final Exams How can the final exams at a university be scheduled so that no 
student has two exams at the same time? 


Solution: This scheduling problem can be solved using a graph model, with vertices representing 
courses and with an edge between two vertices if there is a common student in the courses they 
represent. Each time slot for a final exam is represented by a different color. A scheduling of 
the exams corresponds to a coloring of the associated graph. 

For instance, suppose there are seven finals to be scheduled. Suppose the courses are num- 
bered | through 7. Suppose that the following pairs of courses have common students: | and 2, 
1 and 3, 1 and 4, 1 and 7, 2 and 3, 2 and 4, 2 and 5, 2 and 7, 3 and 4, 3 and 6, 3 and 7, 4 and 5, 
4 and 6, 5 and 6, 5 and 7, and 6 and 7. In Figure 8 the graph associated with this set of classes 
is shown. A scheduling consists of a coloring of this graph. 

Because the chromatic number of this graph is 4 (the reader should verify this), four time 
slots are needed. A coloring of the graph using four colors and the associated schedule are shown 
in Figure 9. 
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5 4 


FIGURE 8 The Graph 


5 Green 4 Brown 


FIGURE 9 Using a Coloring to Schedule Final Exams. 


Representing the Scheduling 


of Final Exams. 


EXAMPLE 6 


EXAMPLE 7 


Exercises 


Now consider an application to the assignment of television channels. 


Frequency Assignments Television channels 2 through 13 are assigned to stations in North 
America so that no two stations within 150 miles can operate on the same channel. How can the 
assignment of channels be modeled by graph coloring? 


Solution: Construct a graph by assigning a vertex to each station. Two vertices are connected by 
an edge if they are located within 150 miles of each other. An assignment of channels corresponds 
to a coloring of the graph, where each color represents a different channel. < 


An application of graph coloring to compilers is considered in Example 7. 


Index Registers In efficient compilers the execution of loops is speeded up when frequently 
used variables are stored temporarily in index registers in the central processing unit, instead 
of in regular memory. For a given loop, how many index registers are needed? This problem 
can be addressed using a graph coloring model. To set up the model, let each vertex of a graph 
represent a variable in the loop. There is an edge between two vertices if the variables they 
represent must be stored in index registers at the same time during the execution of the loop. 
Thus, the chromatic number of the graph gives the number of index registers needed, because 
different registers must be assigned to variables when the vertices representing these variables 
are adjacent in the graph. < 


In Exercises 1—4 construct the dual graph for the map shown. 
Then find the number of colors needed to color the map so 
that no two adjacent regions have the same color. 


TAS, 


In Exercises 5—11 find the chromatic number of the given 
graph. 


11. 


12. 


13. 
14. 


15. 
16. 


17. 


18. 


19. 
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e h i n oO 
a J 
m 
d f g 
b c k i 
For the graphs in Exercises 5-11, decide whether it is 


possible to decrease the chromatic number by removing 
a single vertex and all edges incident with it. 


Which graphs have a chromatic number of 1? 


What is the least number of colors needed to color a map 
of the United States? Do not consider adjacent states 
that meet only at a corner. Suppose that Michigan is 
one region. Consider the vertices representing Alaska and 
Hawaii as isolated vertices. 


What is the chromatic number of W,,? 


Show that a simple graph that has a circuit with an odd 
number of vertices in it cannot be colored using two col- 
ors. 


Schedule the final exams for Math 115, Math 116, 
Math 185, Math 195, CS 101, CS 102, CS 273, and 
CS 473, using the fewest number of different time slots, 
if there are no students taking both Math 115 and CS 473, 
both Math 116 and CS 473, both Math 195 and CS 101, 
both Math 195 and CS 102, both Math 115 and Math 116, 
both Math 115 and Math 185, and both Math 185 and 
Math 195, but there are students in every other pair of 
courses. 


How many different channels are needed for six stations 
located at the distances shown in the table, if two sta- 
tions cannot use the same channel when they are within 
150 miles of each other? 


BEA een ease 
C= Pos [es [am [0 Po 
[fos [= [res [as [on foo 
[Pas [as | — [en [am [a0 


[+ [aw [75 [ow | — [vo [20 
[sf [aw [am [0 | — [a 
[6 [w [0 [20 [2m |v | — 


The mathematics department has six committees, each 
meeting once amonth. How many different meeting times 
must be used to ensure that no member is scheduled to 
attend two meetings at the same time if the committees 
are C,; = {Arlinghaus, Brand, Zaslavsky}, C2 = {Brand, 
Lee, Rosen}, C3 = {Arlinghaus, Rosen, Zaslavsky}, 
C4 = {Lee, Rosen, Zaslavsky}, Cs = {Arlinghaus, 
Brand}, and Ce = {Brand, Rosen, Zaslavsky}? 
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20. A zoo wants to set up natural habitats in which to exhibit 
its animals. Unfortunately, some animals will eat some of 
the others when given the opportunity. How can a graph 
model and a coloring be used to determine the number of 
different habitats needed and the placement of the animals 
in these habitats? 


An edge coloring of a graph is an assignment of colors to 
edges so that edges incident with a common vertex are as- 
signed different colors. The edge chromatic number of a 
graph is the smallest number of colors that can be used in an 
edge coloring of the graph. The edge chromatic number of a 
graph G is denoted by x’(G). 


21. Find the edge chromatic number of each of the graphs in 
Exercises 5-11. 


22. Suppose that n devices are on a circuit board and that 
these devices are connected by colored wires. Express 
the number of colors needed for the wires, in terms of the 
edge chromatic number of the graph representing this cir- 
cuit board, under the requirement that the wires leaving 
a particular device must be different colors. Explain your 
answer. 


23. Find the edge chromatic numbers of 
a) Cy, where n > 3. 
b) W,,, where n > 3. 


24. Show that the edge chromatic number of a graph must be 
at least as large as the maximum degree of a vertex of the 
graph. 


25. Show that if G is a graph with n vertices, then no more 
than n/2 edges can be colored the same in an edge color- 

ing of G. 
* 26. Find the edge chromatic number of K,, when n is a posi- 


tive integer. 


27. Seven variables occur in a loop of a computer program. 
The variables and the steps during which they must be 
stored are t: steps | through 6; u: step 2; v: steps 2 through 
4; w: steps 1, 3, and 5; x: steps 1 and 6; y: steps 3 through 
6; and z: steps 4 and 5. How many different index registers 
are needed to store these variables during execution? 


28. What can be said about the chromatic number of a graph 
that has K,, as a subgraph? 


This algorithm can be used to color a simple graph: First, list 
the vertices vj, v2, V3,..., V, in order of decreasing degree 
so that deg(v,) > deg(v2) > --- > deg(v,,). Assign color 1 to 
vy, and to the next vertex in the list not adjacent to v; (if one 
exists), and successively to each vertex in the list not adjacent 
to a vertex already assigned color 1. Then assign color 2 to 
the first vertex in the list not already colored. Successively 
assign color 2 to vertices in the list that have not already been 
colored and are not adjacent to vertices assigned color 2. If 
uncolored vertices remain, assign color 3 to the first vertex in 
the list not yet colored, and use color 3 to successively color 
those vertices not already colored and not adjacent to vertices 
assigned color 3. Continue this process until all vertices are 
colored. 


29. Construct a coloring of the graph shown using this algo- 
rithm. 


* 30. Use pseudocode to describe this coloring algorithm. 


* 31. Show that the coloring produced by this algorithm may 
use more colors than are necessary to color a graph. 


A connected graph G is called chromatically k-critical if the 
chromatic number of G is k, but for every edge of G, the 
chromatic number of the graph obtained by deleting this edge 
from G isk — 1. 


32. Show that C,, is chromatically 3-critical whenever n is an 
odd positive integer, n > 3. 


33. Show that W,, is chromatically 4-critical whenever n is 
an odd integer, n > 3. 


34. Show that W4 is not chromatically 3-critical. 


35. Show that if G is a chromatically k-critical graph, then 
the degree of every vertex of G is at least k — 1. 


A k-tuple coloring of a graph G is an assignment of a set 
of k different colors to each of the vertices of G such that no 
two adjacent vertices are assigned a common color. We de- 
note by x;(G) the smallest positive integer n such that G has 
a k-tuple coloring using n colors. For example, x2(C4) = 4. 
To see this, note that using only four colors we can assign two 
colors to each vertex of C4, as illustrated, so that no two ad- 
jacent vertices are assigned the same color. Furthermore, no 
fewer than four colors suffice because the vertices vj and v2 
each must be assigned two colors, and a common color cannot 
be assigned to both vy and v2. (For more information about 
k-tuple coloring, see [MiRo91].) 


{red, blue} v; V> {green, yellow} 


{green, yellow} v4 v3 {red, blue} 

36. Find these values: 
a) x2(K3) b) x2(K4) ©) xX2(W4) 
d) x2(Cs) e) xX2(K3,4) f) x3(Ks) 
*g) x3(Cs) h) x3(K4,5) 


* 37, Let G and H be the graphs displayed in Figure 3. Find 


a) x2(G). b) x2(4). 
c) x3(G). d) x3(H). 

38. What is x, (G) if G is a bipartite graph and k is a positive 
integer? 


39. Frequencies for mobile radio (or cellular) telephones are 
assigned by zones. Each zone is assigned a set of fre- 
quencies to be used by vehicles in that zone. The same 
frequency cannot be used in different zones when inter- 
ference can occur between telephones in these zones. Ex- 
plain how a k-tuple coloring can be used to assign k fre- 
quencies to each mobile radio zone in a region. 

* 40. Show that every planar graph G can be colored using six 
or fewer colors. [Hint: Use mathematical induction on 
the number of vertices of the graph. Apply Corollary 2 of 
Section 10.7 to find a vertex v with deg(v) < 5. Consider 
the subgraph of G obtained by deleting v and all edges 
incident with it.] 

** 41, Show that every planar graph G can be colored using five 
or fewer colors. [Hint: Use the hint provided for Exer- 
cise 40.] 

The famous Art Gallery Problem asks how many guards are 

needed to see all parts of an art gallery, where the gallery is the 

interior and boundary of a polygon with n sides. To state this 

problem more precisely, we need some terminology. A point x 

inside or on the boundary of a simple polygon P covers or 

sees a point y inside or on P if all points on the line segment xy 
are in the interior or on the boundary of P. We say that a set 
of points is a guarding set of a simple polygon P if for every 
point y inside P or on the boundary of P there is a point x in 
this guarding set that sees y. Denote by G(P) the minimum 
number of points needed to guard the simple polygon P. The 
art gallery problem asks for the function g(n), which is the 
maximum value of G(P) over all simple polygons with n ver- 
tices. That is, g(”) is the minimum positive integer for which 


Key Terms and Results 


TERMS 


undirected edge: an edge associated to a set {u, v}, where u 
and v are vertices 

directed edge: an edge associated to an ordered pair (u, v), 
where u and v are vertices 

multiple edges: distinct edges connecting the same vertices 

multiple directed edges: distinct directed edges associated 
with the same ordered pair (u,v), where u and v are vertices 

loop: an edge connecting a vertex with itself 

undirected graph: a set of vertices and a set of undirected 
edges each of which is associated with a set of one or two 
of these vertices 

simple graph: an undirected graph with no multiple edges or 
loops 

multigraph: an undirected graph that may contain multiple 
edges but no loops 
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it is guaranteed that a simple polygon with n vertices can be 
guarded with g(n) or fewer guards. 


42. Show that ¢(3) = 1 and g(4) = 1 by showing that all tri- 
angles and quadrilaterals can be guarded using one point. 


* 43. Show that 9(5) = 1. That is, show that all pentagons can 
be guarded using one point. [Hint: Show that there are ei- 
ther 0, 1, or 2 vertices with an interior angle greater than 
180 degrees and that in each case, one guard suffices. ] 


*44, Show that g(6) = 2 by first using Exercises 42 and 43 as 
well as Lemma 1 in Section 5.2 to show that g(6) < 2 
and then find a simple hexagon for which two guards are 
needed. 


* 45, Show that g(n) > |n/3]. [Hint: Consider the polygon 
with 3k vertices that resembles a comb with k prongs, 
such as the polygon with 15 sides shown here.] 


AAAAA 


*46. Solve the art gallery problem by proving the art gallery 
theorem, which states that at most |n/3] guards are 
needed to guard the interior and boundary of a simple 
polygon with n vertices. [Hint: Use Theorem | in Section 
5.2 to triangulate the simple polygon into n — 2 triangles. 
Then show that it is possible to color the vertices of the 
triangulated polygon using three colors so that no two 
adjacent vertices have the same color. Use induction and 
Exercise 23 in Section 5.2. Finally, put guards at all ver- 
tices that are colored red, where red is the color used least 
in the coloring of the vertices. Show that placing guards 
at these points is all that is needed. ] 


pseudograph: an undirected graph that may contain multiple 
edges and loops 


directed graph: a set of vertices together with a set of directed 
edges each of which is associated with an ordered pair of 
vertices 


directed multigraph: a graph with directed edges that may 
contain multiple directed edges 


simple directed graph: a directed graph without loops or mul- 
tiple directed edges 


adjacent: two vertices are adjacent if there is an edge between 
them 


incident: an edge is incident with a vertex if the vertex is an 
endpoint of that edge 

deg v (degree of the vertex v in an undirected graph): the 
number of edges incident with v with loops counted twice 
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deg” (v) (the in-degree of the vertex v in a graph with di- 
rected edges): the number of edges with v as their terminal 
vertex 

degt (v) (the out-degree of the vertex v in a graph with di- 
rected edges): the number of edges with v as their initial 
vertex 

underlying undirected graph of a graph with directed 
edges: the undirected graph obtained by ignoring the di- 
rections of the edges 

K,, (complete graph on n vertices): the undirected graph with 
n vertices where each pair of vertices is connected by an 
edge 

bipartite graph: a graph with vertex set that can be partitioned 
into subsets V; and V2 so that each edge connects a vertex 
in V; and a vertex in V2. The pair (V,, V2) is called a bi- 
partition of V. 

Km,n (complete bipartite graph): the graph with vertex set 
partitioned into a subset of m elements and a subset of 7 el- 
ements with two vertices connected by an edge if and only 
if one is in the first subset and the other is in the second 
subset 

Cn (cycle of size n), n> 3: the graph with n vertices 
V1, V2,.--, Vn and edges {v1, v2}, {v2, v3}, ..., {Vn—1, Vn}, 
(Vn, Vi} 

W,, (wheel of size n), n > 3: the graph obtained from C,, by 
adding a vertex and edges from this vertex to the original 
vertices in Cy, 

Q, (n-cube), n > 1: the graph that has the 2” bit strings of 
length n as its vertices and edges connecting every pair of 
bit strings that differ by exactly one bit 

matching in a graph G: a set of edges such that no two edges 
have a common endpoint 

complete matching M from V, to V2: a matching such that 
every vertex in Vj is an endpoint of an edge in M 

maximum matching: a matching containing the most edges 
among all matchings in a graph 

isolated vertex: a vertex of degree zero 

pendant vertex: a vertex of degree one 

regular graph: a graph where all vertices have the same degree 


subgraph of a graph G = (V, E): a graph (W, F), where W 
is a subset of V and F is a subset of E 

G1 U Gz (union of G; and G2): the graph (V; U V2, Ei U 
E>), where G; = (Vj, £1) and G2 = (Vo, E2) 

adjacency matrix: a matrix representing a graph using the 
adjacency of vertices 

incidence matrix: a matrix representing a graph using the in- 
cidence of edges and vertices 

isomorphic simple graphs: the simple graphs G; = (Vj, E1) 
and G2 = (V2, E2) are isomorphic if there exists a 
one-to-one correspondence f from V; to V2 such that 
{f(v1), f(v2)} € Eo if and only if {v1, v2} € Ej for all v, 
and v2 in Vj 

invariant for graph isomorphism: a property that isomorphic 
graphs either both have or both do not have 

path from wu to v in an undirected graph: a sequence of 
edges e1, €2,...,@n, where e; is associated to {x;, xj+1} 
fori =0,1,...,n, where x9 = u and X;4) =v 


path from u to vy in a graph with directed edges: a sequence 
of edges e1, €2,..., €n, where e; is associated to (x;, Xj+1) 
fori =0,1,...,n, where x9 = u and X;41 =v 

simple path: a path that does not contain an edge more than 
once 

circuit: a path of length n > 1 that begins and ends at the same 
vertex 

connected graph: an undirected graph with the property that 
there is a path between every pair of vertices 

cut vertex of G: a vertex v such that G — v is disconnected 

cut edge of G: an edge e such that G — e is disconnected 

nonseparable graph: a graph without a cut vertex 

vertex cut of G: a subset V’ of the set of vertices of G such 
that G — V’ is disconnected 

«(G) (the vertex connectivity of G): the size of a smallest 
vertex cut of G 

k-connected graph: a graph that has a vertex connectivity no 
smaller than k 

edge cut of G: a set of edges E’ of G such that G — E’ is 
disconnected 

A(G) (the edge connectivity of G): the size of a smallest edge 
cut of G 

connected component of a graph G: a maximal connected 
subgraph of G 

strongly connected directed graph: a directed graph with the 
property that there is a directed path from every vertex to 
every vertex 

strongly connected component of a directed graph G: a 
maximal strongly connected subgraph of G 

Euler path: a path that contains every edge of a graph exactly 
once 

Euler circuit: a circuit that contains every edge of a graph 
exactly once 

Hamilton path: a path in a graph that passes through each 
vertex exactly once 

Hamilton circuit: a circuit in a graph that passes through each 
vertex exactly once 

weighted graph: a graph with numbers assigned to its edges 

shortest-path problem: the problem of determining the path 
in a weighted graph such that the sum of the weights of 
the edges in this path is a minimum over all paths between 
specified vertices 

traveling salesperson problem: the problem that asks for the 
circuit of shortest total length that visits every vertex of a 
weghted graph exactly once 

planar graph: a graph that can be drawn in the plane with no 
crossings 

regions of a representation of a planar graph: the regions 
the plane is divided into by the planar representation of the 
graph 

elementary subdivision: the removal of an edge {u, v} of an 
undirected graph and the addition of a new vertex w together 
with edges {u, w} and {w, v} 

homeomorphic: two undirected graphs are homeomorphic if 
they can be obtained from the same graph by a sequence of 
elementary subdivisions 

graph coloring: an assignment of colors to the vertices of a 
graph so that no two adjacent vertices have the same color 


chromatic number: the minimum number of colors needed 
in a coloring of a graph 


RESULTS 


The handshaking theorem: If G = (V, E) be an undirected 
graph with m edges, then 2m = )°-y deg(v). 

Hall’s marriage theorem: The bipartite graph G = (V, E) 
with bipartition (V;, V2) has a complete matching from V; 
to V2 if and only if |N(A)| > |A| for all subsets A of V1. 

There is an Euler circuit in a connected multigraph if and only 
if every vertex has even degree. 

There is an Euler path in a connected multigraph if and only if 
at most two vertices have odd degree. 


Review Questions 


1. a) Define a simple graph, a multigraph, a pseudograph, 
a directed graph, and a directed multigraph. 


b) Use an example to show how each of the types of graph 
in part (a) can be used in modeling. For example, ex- 
plain how to model different aspects of a computer 
network or airline routes. 


2. Give at least four examples of how graphs are used in 
modeling. 


3. What is the relationship between the sum of the degrees 
of the vertices in an undirected graph and the number of 
edges in this graph? Explain why this relationship holds. 


4. Why must there be an even number of vertices of odd 
degree in an undirected graph? 


5. What is the relationship between the sum of the in-degrees 
and the sum of the out-degrees of the vertices in a directed 
graph? Explain why this relationship holds. 

6. Describe the following families of graphs. 

a) Ky, the complete graph on n vertices 

b) Km.n, the complete bipartite graph on m and n vertices 
c) Cn, the cycle with n vertices 

d) W,, the wheel of size n 

e) Qn, the n-cube 


7. How many vertices and how many edges are there in each 
of the graphs in the families in Question 6? 
8. a) What is a bipartite graph? 
b) Which of the graphs K,, C,, and W,, are bipartite? 
c) How can you determine whether an undirected graph 
is bipartite? 
9. a) Describe three different methods that can be used to 
represent a graph. 
b) Draw a simple graph with at least five vertices and 
eight edges. Illustrate how it can be represented using 
the methods you described in part (a). 


10. a) What does it mean for two simple graphs to be iso- 
morphic? 
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Dijkstra’s algorithm: a procedure for finding a shortest path 
between two vertices in a weighted graph (see Section 10.6). 


Euler’s formula: r = e — v + 2 where, e, and v are the num- 
ber of regions of a planar representation, the number of 
edges, and the number of vertices, respectively, of a con- 
nected planar graph. 


Kuratowski’s theorem: A graph is nonplanar if and only if it 
contains a subgraph homeomorphic to K3,3 or Ks. (Proof 
beyond scope of this book.) 


The four color theorem: Every planar graph can be colored 
using no more than four colors. (Proof far beyond the scope 
of this book!) 


b) What is meant by an invariant with respect to isomor- 
phism for simple graphs? Give at least five examples 
of such invariants. 

c) Give an example of two graphs that have the same 
numbers of vertices, edges, and degrees of vertices, 
but that are not isomorphic. 

d) Is a set of invariants known that can be used to effi- 
ciently determine whether two simple graphs are iso- 
morphic? 

11. a) What does it mean for a graph to be connected? 

b) What are the connected components of a graph? 


12. a) Explain how an adjacency matrix can be used to rep- 
resent a graph. 

b) How can adjacency matrices be used to determine 
whether a function from the vertex set of a graph G to 
the vertex set of a graph H is an isomorphism? 

c) How can the adjacency matrix of a graph be used to 
determine the number of paths of length r, where r is 
a positive integer, between two vertices of a graph? 

13. a) Define an Euler circuit and an Euler path in an undi- 
rected graph. 

b) Describe the famous K6nigsberg bridge problem and 
explain how to rephrase it in terms of an Euler circuit. 

c) How can it be determined whether an undirected graph 
has an Euler path? 

d) How can it be determined whether an undirected graph 
has an Euler circuit? 

14. a) Define a Hamilton circuit in a simple graph. 

b) Give some properties of a simple graph that imply that 

it does not have a Hamilton circuit. 
15. Give examples of at least two problems that can be solved 
by finding the shortest path in a weighted graph. 
16. a) Describe Dijkstra’s algorithm for finding the shortest 
path in a weighted graph between two vertices. 

b) Draw a weighted graph with at least 10 vertices and 
20 edges. Use Dijkstra’s algorithm to find the shortest 
path between two vertices of your choice in the graph. 
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17. a) What does it mean for a graph to be planar? 


b) Give an example of a nonplanar graph. 


18. a) What is Euler’s formula for connected planar graphs? 


b) How can Euler’s formula for planar graphs be used to 
show that a simple graph is nonplanar? 


19. State Kuratowski’s theorem on the planarity of graphs and 
explain how it characterizes which graphs are planar. 


20. a) Define the chromatic number of a graph. 


Supplementary Exercises 


1. How many edges does a 50-regular graph with 100 ver- 
tices have? 


2. How many nonisomorphic subgraphs does K3 have? 


In Exercises 3-5 determine whether two given graphs are iso- 
morphic. 


3. u U4 
u ug 


2 


1 uy U3 
5 U6 uy 
1 Vv 
V6 v7 
V5 Vg 
4 a) 
uy 
Uy 
uy 
Us 


3 


ug 
uy U3 
UG 


vy 


b) What is the chromatic number of the graph K,, when 
n is a positive integer? 

c) What is the chromatic number of the graph C,, when 
n is an integer greater than 2? 

d) What is the chromatic number of the graph Ky, when 
m and n are positive integers? 

21. State the four color theorem. Are there graphs that cannot 
be colored with four colors? 


22. Explain how graph coloring can be used in modeling. Use 
at least two different examples. 


#5, 


uy 


ug uy 


uz U3 


Ug U4 


Us 


The complete m-partite graph K,,, 15,....n,, has vertices par- 
titioned into m subsets of n,,n2,..., 1, elements each, and 
vertices are adjacent if and only if they are in different subsets 
in the partition. 


6. Draw these graphs. 


a) Ki,23 b) K2,2,2 c) K1,2,2,3 
*7, How many vertices and how many edges does the com- 


plete m-partite graph Ky, .n5,....n, have? 


8. Prove or disprove that there are always two vertices of 
the same degree in a finite multigraph having at least two 
vertices. 


9. Let G = (V, E) be an undirected graph and let A C V 
and B C V. Show that 


a) N(AU B) = N(A) UN(B). 


b) N(AN B) C N(A) NO N(B), and give an example 
where N(AN B) 4 N(A) NO N(B). 


10. Let G = (V, E) be an undirected graph. Show that 
a) |N(v)| < deg(v) for allv € V. 


b) |N(v)| = deg vforallv € V ifand only if G is asimple 
graph. 
Suppose that S$), S2,..., 5, is a collection of subsets of 
a set S where n is a positive integer. A system of dis- 
tinct representatives (SDR) for this family is an ordered 
n-tuple (a1, a2,...,@n) with the property that a; € S; for 
i=1,2,...,n anda; #4; foralli F j. 


11. Find a SDR for the sets S; = {a,c,m,e}, So= 
{m,a,c,e}, S3 ={a, p,e,x}, S4={x,e,n,a}, S5= 
{n, a, m, e}, and Sg = {e, x, a, m}. 


12. Use Hall’s marriage theorem to show that a collection 
of finite subsets S,, S2,...,.$, of a set S has a SDR 
(a1, 42,..., Gn) if and only if | Uier Si| = |7| for all sub- 


sets J of {1,2,..., n}. 


13. a) Use Exercise 12 to show that the collection of sets 
Si = {a,b,c}, Sz = {b, c,d}, S83 = {a,b,d}, Sg= 


{b, c, d} has a SDR without finding one explicitly. 
b) Find a SDR for the family of four sets in part (a). 


14. Use Exercise 12 to show that collection of sets S$; = 
{a, b,c}, So = {a,c}, S83 = {c, d, e}, S4 = {b,c}, Ss = 
{d,e, f}, Se = {a, c, e}, and $7 = {a, b} does not have 
a SDR. 

The clustering coefficient C(G) of a simple graph G is the 
probability that if u and v are neighbors and v and w are neigh- 
bors, then uw and w are neighbors, where u, v, and w are distinct 
vertices of G. 


15. We say that three vertices u, v, and w of a simple graph 
G form a triangle if there are edges connecting all three 
pairs of these vertices. Find a formula for C(G) in terms 
of the number of triangles in G and the number of paths 
of length two in the graph. [Hint: Count each triangle 
in the graph once for each order of three vertices that 
form it.] 


16. Find the clustering coefficient of each of the graphs in 


Exercise 20 of Section 10.2 


17. Explain what the clustering coefficient measures in each 


of these graphs. 
a) the Hollywood graph 


b) the graph of Facebook friends 


c) the academic collaboration graph for researchers in 
graph theory 


d) the protein interaction graph for a human cell 


e) the graph representing the routers and communica- 


tions links that make up the worldwide Internet 
18. For each of the graphs in Exercise 17, explain whether 
you would expect its clustering coefficient to be closer to 


0.01 or to 0.10 and why you expect this. 
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A clique in a simple undirected graph is a complete subgraph 
that is not contained in any larger complete subgraph. In Ex- 
ercises 19—21 find all cliques in the graph shown. 


19. a b c 


21. 


A dominating set of vertices in a simple graph is a set of 
vertices such that every other vertex is adjacent to at least one 
vertex of this set. A dominating set with the least number of 
vertices is called a minimum dominating set. In Exercises 
22-24 find a minimum dominating set for the given graph. 


22. 23. 


a b a e 
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A simple graph can be used to determine the minimum number 
of queens on a chessboard that control the entire chessboard. 
Ann x nchessboard has n” squares in ann x n configuration. 
A queen in a given position controls all squares in the same 
row, the same column, and on the two diagonals containing 
this square, as illustrated. The appropriate simple graph has 
n2 vertices, one for each square, and two vertices are adjacent 
if a queen in the square represented by one of the vertices 
controls the square represented by the other vertex. 


The Squares 
Controlled 
by a Queen 


25. Construct the simple graph representing the n x n chess- 
board with edges representing the control of squares by 
queens for 


a) n=3. b) n=4. 


26. Explain how the concept of a minimum dominating set 
applies to the problem of determining the minimum num- 
ber of queens controlling ann x n chessboard. 


** 27, Find the minimum number of queens controlling ann x n 
chessboard for 


a) n=3. b) n=4. c) n=5. 


28. Suppose that G; and Hj are isomorphic and that G2 and 
H are isomorphic. Prove or disprove that G; U G2 and 
HA U Hp are isomorphic. 


29. Show that each of these properties is an invariant that iso- 
morphic simple graphs either both have or both do not 
have. 


a) connectedness 

b) the existence of a Hamilton circuit 
c) the existence of an Euler circuit 
d) having crossing number C 

e) having n isolated vertices 

f) being bipartite 


30. How can the adjacency matrix of G be found from the 
adjacency matrix of G, where G is a simple graph? 


31. How many nonisomorphic connected bipartite simple 
graphs are there with four vertices? 


* 32. How many nonisomorphic simple connected graphs with 
five vertices are there 


a) with no vertex of degree more than two? 


b) with chromatic number equal to four? 
c) that are nonplanar? 


A directed graph is self-converse if it is isomorphic to its 
converse. 


33. Determine whether the following graphs are self- 
converse. 


a) a b 


34. Show that if the directed graph G is self-converse and 
His a directed graph isomorphic to G, then H is also 
self-converse. 


An orientation of an undirected simple graph is an assign- 
ment of directions to its edges such that the resulting di- 
rected graph is strongly connected. When an orientation of 
an undirected graph exists, this graph is called orientable. In 
Exercises 35-37 determine whether the given simple graph is 
orientable. 


35. 4 b c 
© 
e d 
36. a f 
b c d e 
37. a b e 
c 
d f 


38. Because traffic is growing heavy in the central part of a 
city, traffic engineers are planning to change all the streets, 
which are currently two-way, into one-way streets. Ex- 
plain how to model this problem. 


* 39. Show that a graph is not orientable if it has a cut edge. 
A tournament is a simple directed graph such that if wu and 
v are distinct vertices in the graph, exactly one of (u, v) and 
(v, uv) is an edge of the graph. 

40. How many different tournaments are there with n ver- 
tices? 

41. What is the sum of the in-degree and out-degree of a ver- 
tex in a tournament? 


* 42. Show that every tournament has a Hamilton path. 


43. Given two chickens in a flock, one of them is dominant. 
This defines the pecking order of the flock. How can a 
tournament be used to model pecking order? 


44. Suppose that a connected graph G has n vertices and ver- 
tex connectivity «(G) =k. Show that G must have at 
least [kn /2] edges. 


A connected graph G = (V, E) with n vertices and m edges 

is said to have optimal connectivity if «(G) = A(G) = 

minycy deg v = 2m/n. 

45. Show that a connected graph with optimal connectivity 
must be regular. 


46. Show these graphs have optimal connectivity. 
a) C, forn > 3 
b) K, forn > 3 
c) K,, forr > 2 


*47, Find the two nonisomorphic simple graphs with six ver- 
tices and nine edges that have optimal connectivity. 


48. Suppose that G is aconnected multigraph with 2k vertices 
of odd degree. Show that there exist k subgraphs that have 
G as their union, where each of these subgraphs has an 
Euler path and where no two of these subgraphs have an 
edge in common. [Hint: Add k edges to the graph con- 
necting pairs of vertices of odd degree and use an Euler 
circuit in this larger graph.] 


In Exercises 49 and 50 we consider a puzzle posed by Petkovié 
in [Pe09] (based on a problem in [AvCh80]). Suppose that 
King Arthur has gathered his 2n knights of the Round Table 
for an important council. Every two knights are either friends 
or enemies, and each knight has no more than n — | enemies 
among the other 2n — 1 knights. The puzzle asks whether 
King Arthur can seat his knights around the Round Table so 
that each knight has two friends for his neighbors. 


49. a) Show that the puzzle can be reduced to determining 
whether there is a Hamilton circuit in the graph in 
which each knight is represented by a vertex and two 
knights are connected in the graph if they are friends. 

b) Answer the question posed in the puzzle. [Hint: Use 
Dirac’s theorem. ] 


50. Suppose that are eight knights Alynore, Bedivere, De- 
gore, Gareth, Kay, Lancelot, Perceval, and Tristan. Their 
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lists of enemies are A (D, G, P), B (K, P, T), D (A, G, 
L), G (A, D, T), K (B, L, P), L (D, K, T), P (A, B, K), 
T (B, G, L), where we have represented each knight by 
the first letter of his name and shown the list of enemies 
of that knight following this first letter. Draw the graph 
representing these eight knight and their friends and find 
a seating arrangement where each knight sits next to two 
friends. 


*51. Let G be a simple graph with n vertices. The bandwidth 
of G, denoted by B(G), is the minimum, over all permuta- 
tions a1, a2, ..., A, of the vertices of G, of max{|i — j| |ai 
and a; are adjacent}. That is, the bandwidth is the mini- 
mum over all listings of the vertices of the maximum dif- 
ference in the indices assigned to adjacent vertices. Find 
the bandwidths of these graphs. 


a) Ks b) Ki 3 c) K23 
d) K33 e) Q3 f) Cs 


*52. The distance between two distinct vertices vj and v2 of a 
© connected simple graph is the length (number of edges) of 
the shortest path between v; and v2. The radius of a graph 
is the minimum over all vertices v of the maximum dis- 
tance from v to another vertex. The diameter of a graph 
is the maximum distance between two distinct vertices. 

Find the radius and diameter of 


a) Ke. b) Kas. c) Q3. 


*53. a) Show that if the diameter of the simple graph G is at 
least four, then the diameter of its complement Gis 
no more than two. 

b) Show that if the diameter of the simple graph G is at 
least three, then the diameter of its complement Gis 
no more than three. 


d) Co. 


*54, Suppose that a multigraph has 2m vertices of odd degree. 
Show that any circuit that contains every edge of the graph 
must contain at least m edges more than once. 


55. Find the second shortest path between the vertices a and 
z in Figure 3 of Section 10.6. 


56. Devise an algorithm for finding the second shortest path 
between two vertices in a simple connected weighted 
graph. 

57. Find the shortest path between the vertices a and z that 
passes through the vertex f in the weighted graph in Ex- 
ercise 3 in Section 10.6. 

58. Devise an algorithm for finding the shortest path between 
two vertices in a simple connected weighted graph that 
passes through a specified third vertex. 


*59. Show that if G is a simple graph with at least 11 vertices, 
then either G or G, the complement of G, is nonplanar. 


eh set of vertices in a graph is called independent if no two 


vertices in the set are adjacent. The independence number of 
a graph is the maximum number of vertices in an independent 
set of vertices for the graph. 


*60. What is the independence number of 


a) K,? b) Cy? c) Qn? d) Kinn? 
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62. 


63. 
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Show that the number of vertices in a simple graph is less 
than or equal to the product of the independence number 
and the chromatic number of the graph. 


Show that the chromatic number of a graph is less than or 
equal to n —i + 1, where n is the number of vertices in 
the graph and i is the independence number of this graph. 

Suppose that to generate a random simple graph with n 

vertices we first choose a real number p with O < p < 1. 

For each of the C(n, 2) pairs of distinct vertices we gen- 

erate a random number x between 0 and 1. If0 < x < p, 

we connect these two vertices with an edge; otherwise 

these vertices are not connected. 

a) What is the probability that a graph with m edges 
where 0 < m < C(n, 2) is generated? 

b) What is the expected number of edges in a randomly 
generated graph with n vertices if each edge is in- 
cluded with probability p? 

c) Show that if p = 1/2 then every simple graph with n 
vertices is equally likely to be generated. 


A property retained whenever additional edges are added to 
a simple graph (without adding vertices) is called monotone 
increasing, and a property that is retained whenever edges are 


Computer Projects 


Write programs with these input and output. 


. Given the vertex pairs associated to the edges of an undi- 


rected graph, find the degree of each vertex. 


. Given the ordered pairs of vertices associated to the edges 


of a directed graph, determine the in-degree and out- 
degree of each vertex. 


. Given the list of edges of a simple graph, determine 


whether the graph is bipartite. 


. Given the vertex pairs associated to the edges of a graph, 


construct an adjacency matrix for the graph. (Produce a 
version that works when loops, multiple edges, or directed 
edges are present.) 


. Given an adjacency matrix of a graph, list the edges of this 


graph and give the number of times each edge appears. 


. Given the vertex pairs associated to the edges of an undi- 


rected graph and the number of times each edge appears, 
construct an incidence matrix for the graph. 


. Given an incidence matrix of an undirected graph, list its 


edges and give the number of times each edge appears. 


. Given a positive integer n, generate a simple graph with n 


vertices by producing an adjacency matrix for the graph 
so that all simple graphs with n vertices are equally likely 
to be generated. 


. Given a positive integer n, generate a simple directed 


graph with n vertices by producing an adjacency matrix 
for the graph so that all simple directed graphs with n 
vertices are equally likely to be generated. 


removed from a simple graph (without removing vertices) is 
called monotone decreasing. 


64. 


65. 


** 66. 


10. 


11. 


*12. 


13. 


*14, 


** 15, 


** 16, 


17. 


For each of these properties, determine whether it is 
monotone increasing and determine whether it is mono- 
tone decreasing. 

a) The graph G is connected. 

b) The graph G is not connected. 

c) The graph G has an Euler circuit. 

d) The graph G has a Hamilton circuit. 

e) The graph G is planar. 

f) The graph G has chromatic number four. 

g) The graph G has radius three. 

h) The graph G has diameter three. 


Show that the graph property P is monotone increasing if 
and only if the graph property Q is monotone decreasing 
where Q is the property of not having property P. 


Suppose that P is a monotone increasing property of sim- 
ple graphs. Show that the probability a random graph with 
n vertices has property P is a monotonic nondecreasing 
function of p, the probability an edge is chosen to be in 
the graph. 


Given the lists of edges of two simple graphs with no 
more than six vertices, determine whether the graphs are 
isomorphic. 

Given an adjacency matrix of a graph and a positive inte- 
ger n, find the number of paths of length n between two 
vertices. (Produce a version that works for directed and 
undirected graphs.) 


Given the list of edges of a simple graph, determine 
whether it is connected and find the number of connected 
components if it is not connected. 


Given the vertex pairs associated to the edges of a multi- 
graph, determine whether it has an Euler circuit and, if 
not, whether it has an Euler path. Construct an Euler path 
or circuit if it exists. 


Given the ordered pairs of vertices associated to the edges 
of a directed multigraph, construct an Euler path or Euler 
circuit, if such a path or circuit exists. 


Given the list of edges of a simple graph, produce a Hamil- 
ton circuit, or determine that the graph does not have such 
a circuit. 


Given the list of edges of a simple graph, produce a Hamil- 
ton path, or determine that the graph does not have such 
a path. 


Given the list of edges and weights of these edges of a 
weighted connected simple graph and two vertices in this 
graph, find the length of a shortest path between them 
using Dijkstra’s algorithm. Also, find a shortest path. 


18. 


19. 


Given the list of edges of an undirected graph, find a color- 
ing of this graph using the algorithm given in the exercise 
set of Section 10.8. 


Given a list of students and the courses that they are en- 


Computations and Explorations 


20. 
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rolled in, construct a schedule of final exams. 

Given the distances between pairs of television stations 
and the minimum allowable distance between stations, 
assign frequencies to these stations. 


Use a computational program or programs you have written to do these exercises. 


. Display all simple graphs with four vertices. 
. Display a full set of nonisomorphic simple graphs with 


six vertices. 


. Display a full set of nonisomorphic directed graphs with 


four vertices. 


. Generate at random 10 different simple graphs each with 


20 vertices so that each such graph is equally likely to be 
generated. 


. Construct a Gray code where the code words are bit strings 


of length six. 


. Construct knight’s tours on chessboards of various sizes. 
. Determine whether each of the graphs you generated in 


Exercise 4 of this set is planar. If you can, determine the 
thickness of each of the graphs that are not planar. 


. Determine whether each of the graphs you generated in 


Exercise 4 of this set is connected. If a graph is not con- 
nected, determine the number of connected components 
of the graph. 


Writing Projects 


Respond to these with essays using outside sources. 


. Describe the origins and development of graph theory 


prior to the year 1900. 


. Discuss the applications of graph theory to the study of 


ecosystems. 


. Discuss the applications of graph theory to sociology and 


psychology. 


. Discuss what can be learned by investigating the proper- 


ties of the Web graph. 


. Explain what community structure is in a graph repre- 


senting a network, such as a social network, a computer 
network, an information network, or a biological network. 
Define what a community in such a graph is, and explain 
what communities represent in graphs representing the 
types of networks listed. 


. Describe some of the algorithms used to detect commu- 


nities in graphs representing networks of the types listed 
in Question 5. 


. Describe algorithms for drawing a graph on paper or on 


a display given the vertices and edges of the graph. What 


10. 


11. 


#* 12, 


* 13. 


#*14, 


10. 


11. 


12. 


. Generate at random simple graphs with 10 vertices. Stop 


when you have constructed one with an Euler circuit. Dis- 
play an Euler circuit in this graph. 


Generate at random simple graphs with 10 vertices. Stop 
when you have constructed one with a Hamilton circuit. 
Display a Hamilton circuit in this graph. 

Find the chromatic number of each of the graphs you 
generated in Exercise 4 of this set. 


Find the shortest path a traveling salesperson can take to 
visit each of the capitals of the 50 states in the United 
States, traveling by air between cities in a straight line. 


Estimate the probability that arandomly generated simple 
graph with n vertices is connected for each positive integer 
n not exceeding ten by generating a set of random simple 
graphs and determining whether each is connected. 


Work on the problem of determining whether the crossing 
number of K7,7 1s 77, 79, or 81. It is known that it equals 
one of these three values. 


considerations arise in drawing a graph so that it has the 
best appearance for understanding its properties? 


. Explain how graph theory can help uncover networks 


of criminals or terrorists by studying relevant social and 
communication networks. 


. What are some of the capabilities that a software tool 


for inputting, displaying, and manipulating graphs should 
have? Which of these capabilities do available tools have? 


Describe some of the algorithms available for determining 
whether two graphs are isomorphic and the computational 
complexity of these algorithms. What is the most efficient 
such algorithm currently known? 


What is the subgraph isomorphism problem and what 
are some of its important applications, including those to 
chemistry, bioinformatics, electronic circuit design, and 
computer vision? 


Explain what the area of graph mining, an important area 
of data mining, is and describe some of the basic tech- 
niques used in graph mining. 
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13. 


14. 


15. 


16. 


17. 


18. 


19, 
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Describe how Euler paths can be used to help determine 
DNA sequences. 


Define de Bruijn sequences and discuss how they arise 
in applications. Explain how de Bruijn sequences can be 
constructed using Euler circuits. 


Describe the Chinese postman problem and explain how 
to solve this problem. 


Describe some of the different conditions that imply that 
a graph has a Hamilton circuit. 


Describe some of the strategies and algorithms used to 
solve the traveling salesperson problem. 


Describe several different algorithms for determining 
whether a graph is planar. What is the computational com- 
plexity of each of these algorithms? 


In modeling, very large scale integration (VLSD) graphs 
are sometimes embedded in a book, with the vertices on 


20. 
21. 


22. 


23. 


24, 
25. 


the spine and the edges on pages. Define the book num- 
ber of a graph and find the book number of various graphs 
including K, for n = 3, 4,5, and 6. 

Discuss the history of the four color theorem. 

Describe the role computers played in the proof of the 
four color theorem. How can we be sure that a proof that 
relies on a computer is correct? 

Describe and compare several different algorithms for 
coloring a graph, in terms of whether they produce a col- 
oring with the least number of colors possible and in terms 
of their complexity. 

Explain how graph multicolorings can be used in a variety 
of different models. 

Describe some of the applications of edge colorings. 
Explain how the theory of random graphs can be used in 
nonconstructive existence proofs of graphs with certain 
properties. 


11.1 Introduction 
to Trees 


11.2 Applications 
of Trees 


11.3 Tree Traversal 


11.4 Spanning 
Trees 


11.5 Minimum 
Spanning 
Trees 


Trees 


A connected graph that contains no simple circuits is called a tree. Trees were used as long 
ago as 1857, when the English mathematician Arthur Cayley used them to count certain 
types of chemical compounds. Since that time, trees have been employed to solve problems in 
a wide variety of disciplines, as the examples in this chapter will show. 

Trees are particularly useful in computer science, where they are employed in a wide range 
of algorithms. For instance, trees are used to construct efficient algorithms for locating items in 
a list. They can be used in algorithms, such as Huffman coding, that construct efficient codes 
saving costs in data transmission and storage. Trees can be used to study games such as checkers 
and chess and can help determine winning strategies for playing these games. Trees can be used 
to model procedures carried out using a sequence of decisions. Constructing these models can 
help determine the computational complexity of algorithms based on a sequence of decisions, 
such as sorting algorithms. 

Procedures for building trees containing every vertex of a graph, including depth-first search 
and breadth-first search, can be used to systematically explore the vertices of a graph. Explor- 
ing the vertices of a graph via depth-first search, also known as backtracking, allows for the 
systematic search for solutions to a wide variety of problems, such as determining how eight 
queens can be placed on a chessboard so that no queen can attack another. 

We can assign weights to the edges of a tree to model many problems. For example, using 
weighted trees we can develop algorithms to construct networks containing the least expensive 
set of telephone lines linking different network nodes. 


Introduction to Trees 


Links 


In Chapter 10 we showed how graphs can be used to model and solve many problems. In 
this chapter we will focus on a particular type of graph called a tree, so named because such 
graphs resemble trees. For example, family trees are graphs that represent genealogical charts. 
Family trees use vertices to represent the members of a family and edges to represent parent— 
child relationships. The family tree of the male members of the Bernoulli family of Swiss 
mathematicians is shown in Figure |. The undirected graph representing a family tree (restricted 
to people of just one gender and with no inbreeding) is an example of a tree. 


Nikolaus 
(1623-1708) 


a 


Jacob I Nikolaus Johann I 
(1654-1705) (1662-1716) (1667-1748) 


Nikolaus I Nikolaus II Daniel Johann II 
(1687-1759) (1695-1726) (1700-1782) (1710-1790) 


/ \ 


Johann HI Jacob II 
(1746-1807) (1759-1789) 


FIGURE 1 The Bernoulli Family of Mathematicians. 745 
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DEFINITION 1 


EXAMPLE 1 


THEOREM 1 


a b a b a b a b 
d c 
c d 
c d c d 
e 7 e f e f e 3 
G, G, G; G4 


FIGURE 2 Examples of Trees and Graphs That Are Not Trees. 


A tree is aconnected undirected graph with no simple circuits. 


Because a tree cannot have a simple circuit, a tree cannot contain multiple edges or loops. 
Therefore any tree must be a simple graph. 


Which of the graphs shown in Figure 2 are trees? 


Solution: G, and G2 are trees, because both are connected graphs with no simple circuits. G3 is 
not a tree because e, b, a, d, e is a simple circuit in this graph. Finally, G4 is not a tree because 
it is not connected. < 


Any connected graph that contains no simple circuits is a tree. What about graphs containing 
no simple circuits that are not necessarily connected? These graphs are called forests and have 
the property that each of their connected components is a tree. Figure 3 displays a forest. 

Trees are often defined as undirected graphs with the property that there is a unique simple 
path between every pair of vertices. Theorem | shows that this alternative definition is equivalent 
to our definition. 


An undirected graph is a tree if and only if there is a unique simple path between any two of 
its vertices. 


This is one graph with three connected components. 


OS 


FIGURE 3 Example of a Forest. 


DEFINITION 2 
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Proof: First assume that T is a tree. Then T is a connected graph with no simple circuits. Let x 
and y be two vertices of T. Because T is connected, by Theorem | of Section 10.4 there is a 
simple path between x and y. Moreover, this path must be unique, for if there were a second 
such path, the path formed by combining the first path from x to y followed by the path from y 
to x obtained by reversing the order of the second path from x to y would form a circuit. This 
implies, using Exercise 59 of Section 10.4, that there is a simple circuit in T. Hence, there is a 
unique simple path between any two vertices of a tree. 

Now assume that there is a unique simple path between any two vertices of a graph T. 
Then T is connected, because there is a path between any two of its vertices. Furthermore, T 
can have no simple circuits. To see that this is true, suppose T had a simple circuit that contained 
the vertices x and y. Then there would be two simple paths between x and y, because the simple 
circuit is made up of a simple path from x to y and a second simple path from y to x. Hence, a 
graph with a unique simple path between any two vertices is a tree. 4 


Rooted Trees 


In many applications of trees, a particular vertex of a tree is designated as the root. Once we 
specify a root, we can assign a direction to each edge as follows. Because there is a unique path 
from the root to each vertex of the graph (by Theorem 1), we direct each edge away from the 
root. Thus, a tree together with its root produces a directed graph called a rooted tree. 


A rooted tree is a tree in which one vertex has been designated as the root and every edge is 
directed away from the root. 


Rooted trees can also be defined recursively. Refer to Section 5.3 to see how this can be done. 
We can change an unrooted tree into a rooted tree by choosing any vertex as the root. Note that 
different choices of the root produce different rooted trees. For instance, Figure 4 displays the 
rooted trees formed by designating a to be the root and c to be the root, respectively, in the 
tree JT. We usually draw a rooted tree with its root at the top of the graph. The arrows indicating 
the directions of the edges in a rooted tree can be omitted, because the choice of root determines 
the directions of the edges. 

The terminology for trees has botanical and genealogical origins. Suppose that T is a rooted 
tree. If v is a vertex in T other than the root, the parent of v is the unique vertex u such that there 
is a directed edge from u to v (the reader should show that such a vertex is unique). When u is 
the parent of v, v is called a child of uw. Vertices with the same parent are called siblings. The 
ancestors of a vertex other than the root are the vertices in the path from the root to this vertex, 
excluding the vertex itself and including the root (that is, its parent, its parent’s parent, and so 
on, until the root is reached). The descendants of a vertex v are those vertices that have v as 


With root a With root c 
a 


FIGURE 4_ A Tree and Rooted Trees Formed by Designating Two Different Roots. 
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DEFINITION 3 
Links 


EXAMPLE 3 


FIGURE 5 A Rooted Tree 7. FIGURE6 The 
Subtree Rooted at g. 


an ancestor. A vertex of a rooted tree is called a leaf if it has no children. Vertices that have 
children are called internal vertices. The root is an internal vertex unless it is the only vertex 
in the graph, in which case it is a leaf. 

If a is a vertex in a tree, the subtree with a as its root is the subgraph of the tree consisting 
of a and its descendants and all edges incident to these descendants. 


In the rooted tree T (with root a) shown in Figure 5, find the parent of c, the children of g, the 
siblings of h, all ancestors of e, all descendants of b, all internal vertices, and all leaves. What 
is the subtree rooted at g? 


Solution: The parent of c is b. The children of g are h,i, and j. The siblings of # are i and j. 
The ancestors of e are c, b, and a. The descendants of b are c, d, and e. The internal vertices 
are a, b,c, g, h, and j. The leaves are d, e, f, i,k, 1, and m. The subtree rooted at g is shown 
in Figure 6. < 


Rooted trees with the property that all of their internal vertices have the same number of 
children are used in many different applications. Later in this chapter we will use such trees to 
study problems involving searching, sorting, and coding. 


A rooted tree is called an m-ary tree if every internal vertex has no more than m children. 
The tree is called a full m-ary tree if every internal vertex has exactly m children. An m-ary 
tree with m = 2 is called a binary tree. 


Are the rooted trees in Figure 7 full m-ary trees for some positive integer m? 


AKA BAK 


FIGURE 7 _ Four Rooted Trees. 


EXAMPLE 4 
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Solution: T, is a full binary tree because each of its internal vertices has two children. T> is a 
full 3-ary tree because each of its internal vertices has three children. In 73 each internal vertex 
has five children, so 73 is a full S-ary tree. 74 is not a full m-ary tree for any m because some of 
its internal vertices have two children and others have three children. 


ORDERED ROOTED TREES An ordered rooted tree is a rooted tree where the children 
of each internal vertex are ordered. Ordered rooted trees are drawn so that the children of each 
internal vertex are shown in order from left to right. Note that a representation of a rooted tree in 
the conventional way determines an ordering for its edges. We will use such orderings of edges 
in drawings without explicitly mentioning that we are considering a rooted tree to be ordered. 

In an ordered binary tree (usually called just a binary tree), if an internal vertex has two 
children, the first child is called the left child and the second child is called the right child. 
The tree rooted at the left child of a vertex is called the left subtree of this vertex, and the tree 
rooted at the right child of a vertex is called the right subtree of the vertex. The reader should 
note that for some applications every vertex of a binary tree, other than the root, is designated 
as aright or a left child of its parent. This is done even when some vertices have only one child. 
We will make such designations whenever it is necessary, but not otherwise. 

Ordered rooted trees can be defined recursively. Binary trees, a type of ordered rooted trees, 
were defined this way in Section 5.3. 


What are the left and right children of d in the binary tree T shown in Figure 8(a) (where the 
order is that implied by the drawing)? What are the left and right subtrees of c? 


Solution: The left child of d is f and the right child is g. We show the left and right subtrees 
of c in Figures 8(b) and 8(c), respectively. < 


> 


a. 
> 


m 


(b) (c) 


FIGURE 8 A Binary Tree T and Left and Right Subtrees of the Vertex c. 


Just as in the case of graphs, there is no standard terminology used to describe trees, rooted 
trees, ordered rooted trees, and binary trees. This nonstandard terminology occurs because trees 
are used extensively throughout computer science, which is a relatively young field. The reader 
should carefully check meanings given to terms dealing with trees whenever they occur. 


Trees as Models 


Trees are used as models in such diverse areas as computer science, chemistry, geology, botany, 
and psychology. We will describe a variety of such models based on trees. 
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EXAMPLE 7 


Links 


H 
H—C—H H H H 
H—C—H H Cc Cc i H 
H—C—H H H 

H—C—H 
H—C—H 
H 
H 
Butane Isobutane 


FIGURE 9 The Two Isomers of Butane. 


Saturated Hydrocarbons and Trees Graphs can be used to represent molecules, where atoms 
are represented by vertices and bonds between them by edges. The English mathematician Arthur 
Cayley discovered trees in 1857 when he was trying to enumerate the isomers of compounds of 
the form C,,H2)+42, which are called saturated hydrocarbons. 

In graph models of saturated hydrocarbons, each carbon atom is represented by a vertex 
of degree 4, and each hydrogen atom is represented by a vertex of degree 1. There are 3n + 2 
vertices in a graph representing a compound of the form C, H2,+2. The number of edges in sucha 
graph is half the sum of the degrees of the vertices. Hence, there are (4n + 2n + 2)/2 = 3n+ 1 
edges in this graph. Because the graph is connected and the number of edges is one less than 
the number of vertices, it must be a tree (see Exercise 15). 

The nonisomorphic trees with n vertices of degree 4 and 2n + 2 of degree | represent the 
different isomers of C,,H2,)42. For instance, when n = 4, there are exactly two nonisomorphic 
trees of this type (the reader should verify this). Hence, there are exactly two different isomers 
of C4Hjo. Their structures are displayed in Figure 9. These two isomers are called butane and 
isobutane. < 


Representing Organizations The structure of a large organization can be modeled using a 
rooted tree. Each vertex in this tree represents a position in the organization. An edge from one 
vertex to another indicates that the person represented by the initial vertex is the (direct) boss 
of the person represented by the terminal vertex. The graph shown in Figure 10 displays such a 
tree. In the organization represented by this tree, the Director of Hardware Development works 
directly for the Vice President of R&D. < 


Computer File Systems Files in computer memory can be organized into directories. A 
directory can contain both files and subdirectories. The root directory contains the entire file 


ARTHUR CAYLEY (1821-1895) Arthur Cayley, the son of a merchant, displayed his mathematical talents 
at an early age with amazing skill in numerical calculations. Cayley entered Trinity College, Cambridge, when 
he was 17. While in college he developed a passion for reading novels. Cayley excelled at Cambridge and 
was elected to a 3-year appointment as Fellow of Trinity and assistant tutor. During this time Cayley began 
his study of n-dimensional geometry and made a variety of contributions to geometry and to analysis. He 
also developed an interest in mountaineering, which he enjoyed during vacations in Switzerland. Because no 
position as a mathematician was available to him, Cayley left Cambridge, entering the legal profession and 
gaining admittance to the bar in 1849. Although Cayley limited his legal work to be able to continue his 
mathematics research, he developed a reputation as a legal specialist. During his legal career he was able to 


write more than 300 mathematical papers. In 1863 Cambridge University established a new post in mathematics and offered it to 
Cayley. He took this job, even though it paid less money than he made as a lawyer. 
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President 
YP VP VP VP 
R&D Marketing Services Finance 
Director cared cae AVP AVP — nee Director Director 
Software Hardware ‘ Field Material : 
Research Sales | | Marketing : Accounting MIS 
Development | | Development Operations Management 


FIGURE 10 An Organizational Tree for a Computer Company. 


EXAMPLE 8 


Py 


Ps 


Ps 


P, 


FIGURE 12 A 
Tree-Connected 
Network of Seven 


Processors. 


system. Thus, a file system may be represented by a rooted tree, where the root represents the 
root directory, internal vertices represent subdirectories, and leaves represent ordinary files or 
empty directories. One such file system is shown in Figure 11. In this system, the file khr is in 
the directory rje. (Note that links to files where the same file may have more than one pathname 
can lead to circuits in computer file systems.) 


The root is the root directory / 
Internal vertices are directories 


Ree ais ‘les 


f 


ae 


usr bin tmp 
bin rje spool Is mail who junk 
JIN | EN 
ed nroff vikhr opr uucp 
printer file 


FIGURE 11 A Computer File System. 


Tree-Connected Parallel Processors In Example 17 of Section 10.2 we described several 
interconnection networks for parallel processing. A tree-connected network is another impor- 
tant way to interconnect processors. The graph representing such a network is a complete binary 
tree, that is, a full binary tree where every root is at the same level. Such a network interconnects 
n = 2" — 1 processors, where k is a positive integer. A processor represented by the vertex v 
that is not a root or a leaf has three two-way connections—one to the processor represented by 
the parent of v and two to the processors represented by the two children of v. The processor 
represented by the root has two two-way connections to the processors represented by its two 
children. A processor represented by a leaf v has a single two-way connection to the parent of v. 
We display a tree-connected network with seven processors in Figure 12. 

We now illustrate how a tree-connected network can be used for parallel computation. In 
particular, we show how the processors in Figure 12 can be used to add eight numbers, using 
three steps. In the first step, we add x; and x2 using P4, x3 and x4 using Ps, x5 and x6 using Pe, 
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and x7 and xg using P7. In the second step, we add x; + x2 and x3 + x4 using P2 and x5 + x6 and 
x7 + xg using P3. Finally, in the third step, we add x; + x2 + x3 + x4 and x5 + x6 +x7 + xg 
using P;. The three steps used to add eight numbers compares favorably to the seven steps 
required to add eight numbers serially, where the steps are the addition of one number to the 
sum of the previous numbers in the list. 


Properties of Trees 


We will often need results relating the numbers of edges and vertices of various types in trees. 


A tree with n vertices has n — | edges. 


Proof: We will use mathematical induction to prove this theorem. Note that for all the trees here 
we can choose a root and consider the tree rooted. 


BASIS STEP: When n = 1, a tree with n = 1 vertex has no edges. It follows that the theorem 
is true forn = 1. 


INDUCTIVE STEP: The inductive hypothesis states that every tree with k vertices has k — 1 
edges, where k is a positive integer. Suppose that a tree T has k + 1 vertices and that v is a 
leaf of T (which must exist because the tree is finite), and let w be the parent of v. Removing 
from T the vertex v and the edge connecting w to v produces a tree T’ with k vertices, because 
the resulting graph is still connected and has no simple circuits. By the inductive hypothesis, T’ 
has k — 1 edges. It follows that T has k edges because it has one more edge than T’, the edge 
connecting v and w. This completes the inductive step. | 


Recall that a tree is a connected undirected graph with no simple circuits. So, when G is an 
undirected graph with n vertices, Theorem 2 tells us that the two conditions (i) G is connected 
and (ii) G has no simple circuits, imply (iii) G has n — | edges. Also, when (i) and (iii) hold, 
then (ii) must also hold, and when (ii) and (iii) hold, (i) must also hold. That is, if G is connected 
and G has n — | edges, then G has no simple circuits, so that G is a tree (see Exercise 15(a)), 
and if G has no simple circuits and G has n — | edges, then G is connected, and so is a tree (see 
Exercise 15(b)). Consequently, when two of (i), (ii), and (iii) hold, the third condition must also 
hold, and G must be a tree. 


COUNTING VERTICES IN FULL m-ARY TREES The number of vertices in a full m-ary 
tree with a specified number of internal vertices is determined, as Theorem 3 shows. As in 
Theorem 2, we will use 1 to denote the number of vertices in a tree. 


A full m-ary tree with i internal vertices contains n = mi + | vertices. 


Proof: Every vertex, except the root, is the child of an internal vertex. Because each of the i 
internal vertices has m children, there are mi vertices in the tree other than the root. Therefore, 
the tree contains n = mi + 1 vertices. <j 


Suppose that 7 is a full m-ary tree. Let i be the number of internal vertices and / the number 
of leaves in this tree. Once one of n, i, and / is known, the other two quantities are determined. 
Theorem 4 explains how to find the other two quantities from the one that is known. 


THEOREM 4 


EXAMPLE 9 


FIGURE 13 A 
Rooted Tree. 


EXAMPLE 10 


EXAMPLE 11 
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A full m-ary tree with 


(i) n vertices has i = (n — 1)/m internal vertices and / = [(m — 1)n + 1]/m leaves, 
(ii) 7 internal vertices has n = mi + 1 vertices and / = (m — 1)i + 1 leaves, 
(iii) | leaves has n = (ml — 1)/(m — 1) vertices and i = (J — 1)/(m — 1) internal ver- 
tices. 


Proof: Let n represent the number of vertices, i the number of internal vertices, and / the number 
of leaves. The three parts of the theorem can all be proved using the equality given in Theorem 3, 
that is, n = mi+ 1, together with the equality n = / +7, which is true because each vertex is 
either a leaf or an internal vertex. We will prove part (7) here. The proofs of parts (ii) and (iii) 
are left as exercises for the reader. 

Solving for i inn = mi+ | gives i = (n — 1)/m. Then inserting this expression for i into 
the equation n = / +i shows that] =n —i =n—(n—1)/m =[(m— 1)n+1]/m. 4 


Example 9 illustrates how Theorem 4 can be used. 


Suppose that someone starts a chain letter. Each person who receives the letter is asked to send 
it on to four other people. Some people do this, but others do not send any letters. How many 
people have seen the letter, including the first person, if no one receives more than one letter 
and if the chain letter ends after there have been 100 people who read it but did not send it out? 
How many people sent out the letter? 


Solution: The chain letter can be represented using a 4-ary tree. The internal vertices correspond 
to people who sent out the letter, and the leaves correspond to people who did not send it 
out. Because 100 people did not send out the letter, the number of leaves in this rooted tree is 
1 = 100. Hence, part (iii) of Theorem 4 shows that the number of people who have seen the letter 
isn = (4- 100 — 1)/(4 — 1) = 133. Also, the number of internal vertices is 133 — 100 = 33, 
so 33 people sent out the letter. < 


BALANCED m-ARY TREES | It is often desirable to use rooted trees that are “balanced” so 
that the subtrees at each vertex contain paths of approximately the same length. Some definitions 
will make this concept clear. The level of a vertex v in a rooted tree is the length of the unique 
path from the root to this vertex. The level of the root is defined to be zero. The height of a 
rooted tree is the maximum of the levels of vertices. In other words, the height of a rooted tree 
is the length of the longest path from the root to any vertex. 


Find the level of each vertex in the rooted tree shown in Figure 13. What is the height of this 
tree? 


Solution: The root a is at level 0. Vertices b, 7, and k are at level 1. Vertices c, e, f, and / are at 
level 2. Vertices d, g,i, m, and n are at level 3. Finally, vertex h is at level 4. Because the largest 
level of any vertex is 4, this tree has height 4. < 


A rooted m-ary tree of height h is balanced if all leaves are at levels h or h — 1. 
Which of the rooted trees shown in Figure 14 are balanced? 
Solution: T, is balanced, because all its leaves are at levels 3 and 4. However, T> is not balanced, 


because it has leaves at levels 2, 3, and 4. Finally, 7; is balanced, because all its leaves are at 
level 3. < 
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FIGURE 14 Some Rooted Trees. 
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COROLLARY 1 


ABOUND FORTHE NUMBER OF LEAVES IN AN m-ARYTREE _Itis often useful to have 
an upper bound for the number of leaves in an m-ary tree. Theorem 5 provides such a bound in 
terms of the height of the m-ary tree. 


There are at most m! leaves in an m-ary tree of height h. 


Proof: The proof uses mathematical induction on the height. First, consider m-ary trees of 
height 1. These trees consist of a root with no more than m children, each of which is a leaf. 
Hence, there are no more than m! = m leaves in an m-ary tree of height 1. This is the basis step 
of the inductive argument. 

Now assume that the result is true for all m-ary trees of height less than h; this is the inductive 
hypothesis. Let T be an m-ary tree of height h. The leaves of T are the leaves of the subtrees 
of T obtained by deleting the edges from the root to each of the vertices at level 1, as shown in 
Figure 15. 

Each of these subtrees has height less than or equal to h — 1. So by the inductive hypothesis, 
each of these rooted trees has at most m!'—! leaves. Because there are at most m such subtrees, 
each with a maximum of m’—! leaves, there are at most m-m"—! = m" leaves in the rooted 
tree. This finishes the inductive argument. 4 


If an m-ary tree of height / has / leaves, then h > [log,,/]. If the m-ary tree is full and 
balanced, then h = [log,, /]. (We are using the ceiling function here. Recall that [x] is the 
smallest integer greater than or equal to x.) 


Proof: We know that / < m" from Theorem 5. Taking logarithms to the base m shows that 
log,, | < h. Because h is an integer, we have h > [log,,, /]. Now suppose that the tree is balanced. 


Ist 2nd 3rd (m — 1)st mth 
subtree subtree subtree subtree subtree 
of height of height of height of height of height 
a | =nr-] =Sh-] Sk=1 =k-1 


FIGURE 15 _ The Inductive Step of the Proof. 
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Then each leaf is at level h or h — 1, and because the height is h, there is at least one leaf at level h. 
It follows that there must be more than m"~! leaves (see Exercise 30). Because 1 < m", we have 


h-1 


m <1 <m". Taking logarithms to the base m in this inequality gives h — 1 < log, ) <h. 


Hence, h = [log,, /]. 


Exercises 


1. Which of these graphs are trees? 


a) YJ] b) Y~ 
e 
S| | aa 
; 4 
2. Which of these graphs are trees? 
x , Wave 
5 
e) f) 6 
7 
8 
9 
3. Answer these questions about the rooted tree illustrated. 
a 
10. 


11. 


#12, 


a) Which vertex is the root? 

b) Which vertices are internal? 

c) Which vertices are leaves? 

d) Which vertices are children of j? 

e) Which vertex is the parent of h? 

f) Which vertices are siblings of 0? 

g) Which vertices are ancestors of m? 
h) Which vertices are descendants of b? 


. Answer the same questions as listed in Exercise 3 for the 


rooted tree illustrated. 


. Is the rooted tree in Exercise 3 a full m-ary tree for some 


positive integer m? 


. Is the rooted tree in Exercise 4 a full m-ary tree for some 


positive integer m? 


. What is the level of each vertex of the rooted tree in Ex- 


ercise 3? 


. What is the level of each vertex of the rooted tree in Ex- 


ercise 4? 


. Draw the subtree of the tree in Exercise 3 that is rooted 


at 
a) a. b) c. c) e. 


Draw the subtree of the tree in Exercise 4 that is rooted 

at 

a) a. b) c. c) e. 

a) How many nonisomorphic unrooted trees are there 
with three vertices? 

b) How many nonisomorphic rooted trees are there 
with three vertices (using isomorphism for directed 
graphs)? 

a) How many nonisomorphic unrooted trees are there 
with four vertices? 

b) How many nonisomorphic rooted trees are there 
with four vertices (using isomorphism for directed 
graphs)? 
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*13. a) How many nonisomorphic unrooted trees are there 


*14, 


Tsr# 15 


with five vertices? 


b) How many nonisomorphic rooted trees are there with 
five vertices (using isomorphism for directed graphs)? 


Show that a simple graph is a tree if and only if it is 
connected but the deletion of any of its edges produces a 
graph that is not connected. 


Let G be a simple graph with n vertices. Show that 


a) Gis atree if and only if it is connected and has n — 1 
edges. 


b) G is atree if and only if G has no simple circuits and 
has n — 1 edges. [Hint: To show that G is connected 
if it has no simple circuits and n — 1 edges, show that 
G cannot have more than one connected component. ] 


16. Which complete bipartite graphs Ky,.,, where m and n 


17 
18 


19, 


20 


21 


are positive integers, are trees? 
How many edges does a tree with 10,000 vertices have? 


How many vertices does a full 5-ary tree with 100 internal 
vertices have? 


How many edges does a full binary tree with 1000 internal 
vertices have? 


How many leaves does a full 3-ary tree with 100 vertices 
have? 


Suppose 1000 people enter a chess tournament. Use a 
rooted tree model of the tournament to determine how 
many games must be played to determine a champion, if 
a player is eliminated after one loss and games are played 
until only one entrant has not lost. (Assume there are no 
ties.) 


22. A chain letter starts when a person sends a letter to five 


others. Each person who receives the letter either sends it 
to five other people who have never received it or does not 
send it to anyone. Suppose that 10,000 people send out 
the letter before the chain ends and that no one receives 
more than one letter. How many people receive the letter, 
and how many do not send it out? 


23. A chain letter starts with a person sending a letter out 


* 24, 


*25 


to 10 others. Each person is asked to send the letter out 
to 10 others, and each letter contains a list of the previous 
six people in the chain. Unless there are fewer than six 
names in the list, each person sends one dollar to the first 
person in this list, removes the name of this person from 
the list, moves up each of the other five names one posi- 
tion, and inserts his or her name at the end of this list. If 
no person breaks the chain and no one receives more than 
one letter, how much money will a person in the chain 
ultimately receive? 


Either draw a full m-ary tree with 76 leaves and height 3, 
where m is a positive integer, or show that no such tree 
exists. 


Either draw a full m-ary tree with 84 leaves and height 3, 
where m is a positive integer, or show that no such tree 
exists. 


* 26. A full m-ary tree T has 81 leaves and height 4. 


a) Give the upper and lower bounds for m. 
b) What is m if T is also balanced? 


A complete m-ary tree is a full m-ary tree in which every leaf 
is at the same level. 


27. 


28. 


29. 


~ 30. 


31. 


32 


33. 


34. 


35. 


36. 


37. 


Construct a complete binary tree of height 4 and a com- 
plete 3-ary tree of height 3. 


How many vertices and how many leaves does a complete 
m-ary tree of height h have? 

Prove 

a) part (ii) of Theorem 4. 

b) part (iii) of Theorem 4. 

Show that a full m-ary balanced tree of height / has more 
than m"—! leaves. 


How many edges are there in a forest of t trees containing 
a total of n vertices? 


Explain how a tree can be used to represent the table of 
contents of a book organized into chapters, where each 
chapter is organized into sections, and each section is or- 
ganized into subsections. 


How many different isomers do these saturated hydro- 
carbons have? 


a) C3H¢ b) CsHi2 c) CoHi4 
What does each of these represent in an organizational 
tree? 


a) the parent of a vertex 

b) achild of a vertex 

c) a sibling of a vertex 

d) the ancestors of a vertex 

e) the descendants of a vertex 
f) the level of a vertex 

g) the height of the tree 


Answer the same questions as those given in Exercise 34 
for a rooted tree representing a computer file system. 


a) Draw the complete binary tree with 15 vertices that 
represents a tree-connected network of 15 processors. 


b) Show how 16 numbers can be added using the 15 pro- 
cessors in part (a) using four steps. 


Let n be a power of 2. Show that n numbers can be added 
in logn steps using a tree-connected network of n — 1 
processors. 


* 38. A labeled tree is a tree where each vertex is assigned a 


label. Two labeled trees are considered isomorphic when 
there is an isomorphism between them that preserves the 
labels of vertices. How many nonisomorphic trees are 
there with three vertices labeled with different integers 
from the set {1, 2,3}? How many nonisomorphic trees 
are there with four vertices labeled with different inte- 
gers from the set {1, 2, 3, 4}? 
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The eccentricity of a vertex in an unrooted tree is the length 42. Show that a center should be chosen as the root to produce 
of the longest simple path beginning at this vertex. A vertex is a rooted tree of minimal height from an unrooted tree. 
called a center if no vertex in the tree has smaller eccentricity * 43. Show that a tree has either one center or two centers that 
than this vertex. In Exercises 39-41 find every vertex that is a are adjacent. 


center in the given tree. 


39. 


44. Show that every tree can be colored using two colors. 


The rooted Fibonacci trees 7,, are defined recursively in the 
following way. T; and T> are both the rooted tree consisting 
of a single vertex, and for n = 3,4,..., the rooted tree T,, is 
constructed from a root with T,,__; as its left subtree and T,,_2 
as its right subtree. 


45. Draw the first seven rooted Fibonacci trees. 


* 46. How many vertices, leaves, and internal vertices does the 
rooted Fibonacci tree T,, have, where n is a positive inte- 
ger? What is its height? 

47. What is wrong with the following “proof” using mathe- 
matical induction of the statement that every tree with n 
vertices has a path of length n — 1. Basis step: Every tree 
with one vertex clearly has a path of length 0. Inductive 
step: Assume that a tree with n vertices has a path of 
length n — 1, which has wu as its terminal vertex. Add a 
vertex v and the edge from u to v. The resulting tree has 
n + | vertices and has a path of length n. This completes 
the inductive step. 


Cs** 48. Show that the average depth of a leaf in a binary tree 
with n vertices is Q(logn). 


Applications of Trees 


Links 


Introduction 


We will discuss three problems that can be studied using trees. The first problem is: How should 
items in a list be stored so that an item can be easily located? The second problem is: What 
series of decisions should be made to find an object with a certain property in a collection of 
objects of a certain type? The third problem is: How should a set of characters be efficiently 
coded by bit strings? 


Binary Search Trees 


Searching for items in a list is one of the most important tasks that arises in computer science. 
Our primary goal is to implement a searching algorithm that finds items efficiently when the 
items are totally ordered. This can be accomplished through the use of a binary search tree, 
which is a binary tree in which each child of a vertex is designated as a right or left child, no 
vertex has more than one right child or left child, and each vertex is labeled with a key, which 
is one of the items. Furthermore, vertices are assigned keys so that the key of a vertex is both 
larger than the keys of all vertices in its left subtree and smaller than the keys of all vertices in 
its right subtree. 

This recursive procedure is used to form the binary search tree for a list of items. Start with 
a tree containing just one vertex, namely, the root. The first item in the list is assigned as the 
key of the root. To add a new item, first compare it with the keys of vertices already in the tree, 
starting at the root and moving to the left if the item is less than the key of the respective vertex 
if this vertex has a left child, or moving to the right if the item is greater than the key of the 
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respective vertex if this vertex has a right child. When the item is less than the respective vertex 
and this vertex has no left child, then a new vertex with this item as its key is inserted as a 
new left child. Similarly, when the item is greater than the respective vertex and this vertex has 
no right child, then a new vertex with this item as its key is inserted as a new right child. We 
illustrate this procedure with Example 1. 


Form a binary search tree for the words mathematics, physics, geography, zoology, meteorology, 
geology, psychology, and chemistry (using alphabetical order). 


Solution: Figure 1 displays the steps used to construct this binary search tree. The word mathe- 
matics is the key of the root. Because physics comes after mathematics (in alphabetical order), 
add a right child of the root with key physics. Because geography comes before mathemat- 
ics, add a left child of the root with key geography. Next, add a right child of the vertex with 
key physics, and assign it the key zoology, because zoology comes after mathematics and after 
physics. Similarly, add a left child of the vertex with key physics and assign this new vertex the 
key meteorology. Add a right child of the vertex with key geography and assign this new vertex 
the key geology. Add a left child of the vertex with key zoology and assign it the key psychology. 
Add a left child of the vertex with key geography and assign it the key chemistry. (The reader 
should work through all the comparisons needed at each step.) < 


Once we have a binary search tree, we need a way to locate items in the binary search tree, 
as well as a way to add new items. Algorithm 1, an insertion algorithm, actually does both of 
these tasks, even though it may appear that it is only designed to add vertices to a binary search 
tree. That is, Algorithm | is a procedure that locates an item x in a binary search tree if it is 
present, and adds a new vertex with x as its key if x is not present. In the pseudocode, v is the 
vertex currently under examination and /abel(v) represents the key of this vertex. The algorithm 
begins by examining the root. If x equals the key of v, then the algorithm has found the location 
of x and terminates; if x is less than the key of v, we move to the left child of v and repeat the 
procedure; and if x is greater than the key of v, we move to the right child of v and repeat the 
procedure. If at any step we attempt to move to a child that is not present, we know that x is not 
present in the tree, and we add a new vertex as this child with x as its key. 


mathematics 
e 


mathematics mathematics mathematics 


physics 
physics geography _ physics geography 


zoology 


zoology > mathematics 
physics > mathematics geography < mathematics zoology > physics 


mathematics 


physics a physics 
geography 


meteorology zoology meteorology meteorology 


meteorology > mathematics geology < mathematics psychology > physics chemistry < mathematics 
meteorology < physics 


mathematics mathematics mathematics 


geography geography 
= physics geology ® Physics 


chemistry ca 


psychology meteorology psychology 
psychology > mathematics 


geology zoology geology zoology zoology 


geology > geography psychology < zoology chemistry < geography 


FIGURE 1 Constructing a Binary Search Tree. 
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ALGORITHM 1 Locating an Item in or Adding an Item to a Binary Search Tree. 


procedure insertion(T: binary search tree, x: item) 
v := root of T 
{a vertex not present in T has the value null } 
while v ¥ null and label(v) 4 x 
if x < label(v) then 
if left child of v A null then v := left child of v 
else add new vertex as a left child of v and set v := null 
else 
if right child of v A null then v := right child of v 
else add new vertex as a right child of v and set v := null 
if root of T = null then add a vertex v to the tree and label it with x 
else if v is null or /abel(v) € x then label new vertex with x and let v be this new vertex 
return v {v = location of x} 


Example 2 illustrates the use of Algorithm | to insert a new item into a binary search tree. 


Use Algorithm | to insert the word oceanography into the binary search tree in Example 1. 


Solution: Algorithm | begins with v, the vertex under examination, equal to the root of T, so 
label(v) = mathematics. Because v 4 null and label(v) = mathematics < oceanography, we 
next examine the right child of the root. This right child exists, so we set v, the vertex under 
examination, to be this right child. At this step we have v 4 null and label(v) = physics > 
oceanography, so we examine the left child of v. This left child exists, so we set v, the vertex under 
examination, to this left child. At this step, we also have v 4 null and label(v) = metereology < 
oceanography, so we try to examine the right child of v. However, this right child does not exist, 
so we add a new vertex as the right child of v (which at this point is the vertex with the key 
metereology) and we set v := null. We now exit the while loop because v = null. Because the 
root of T is not null and v = null, we use the else if statement at the end of the algorithm to 
label our new vertex with the key oceanography. < 


We will now determine the computational complexity of this procedure. Suppose we have 
a binary search tree T for a list of n items. We can form a full binary tree U from T by adding 
unlabeled vertices whenever necessary so that every vertex with a key has two children. This is 
illustrated in Figure 2. Once we have done this, we can easily locate or add a new item as a key 
without adding a vertex. 

The most comparisons needed to add a new item is the length of the longest path in U from 
the root to a leaf. The internal vertices of U are the vertices of T. It follows that U has n internal 
vertices. We can now use part (ii) of Theorem 4 in Section 11.1 to conclude that U has n + 1 
leaves. Using Corollary | of Section 11.1, we see that the height of U is greater than or equal to 
h = [log(n + 1)]. Consequently, it is necessary to perform at least [log(n + 1)] comparisons 
to add some item. Note that if U is balanced, its height is [log(n + 1)] (by Corollary | of 
Section 11.1). Thus, if a binary search tree is balanced, locating or adding an item requires no 
more than [log(n + 1)] comparisons. A binary search tree can become unbalanced as items 
are added to it. Because balanced binary search trees give optimal worst-case complexity for 
binary searching, algorithms have been devised that rebalance binary search trees as items are 
added. The interested reader can consult references on data structures for the description of such 
algorithms. 
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FIGURE 2 Adding Unlabeled Vertices to Make a Binary Search Tree Full. 
Decision Trees 


Rooted trees can be used to model problems in which a series of decisions leads to a solution. 
For instance, a binary search tree can be used to locate items based on a series of comparisons, 
where each comparison tells us whether we have located the item, or whether we should go 
right or left in a subtree. A rooted tree in which each internal vertex corresponds to a decision, 
with a subtree at these vertices for each possible outcome of the decision, is called a decision 
tree. The possible solutions of the problem correspond to the paths to the leaves of this rooted 
tree. Example 3 illustrates an application of decision trees. 


Suppose there are seven coins, all with the same weight, and a counterfeit coin that weighs less 
than the others. How many weighings are necessary using a balance scale to determine which 
of the eight coins is the counterfeit one? Give an algorithm for finding this counterfeit coin. 


Solution: There are three possibilities for each weighing on a balance scale. The two pans can 
have equal weight, the first pan can be heavier, or the second pan can be heavier. Consequently, 
the decision tree for the sequence of weighings is a 3-ary tree. There are at least eight leaves in 
the decision tree because there are eight possible outcomes (because each of the eight coins can 
be the counterfeit lighter coin), and each possible outcome must be represented by at least one 
leaf. The largest number of weighings needed to determine the counterfeit coin is the height of 
the decision tree. From Corollary | of Section 11.1 it follows that the height of the decision tree 
is at least [log 8] = 2. Hence, at least two weighings are needed. 

It is possible to determine the counterfeit coin using two weighings. The decision tree that 
illustrates how this is done is shown in Figure 3. < 


THE COMPLEXITY OF COMPARISON-BASED SORTING ALGORITHMS Many dif- 
ferent sorting algorithms have been developed. To decide whether a particular sorting algorithm 
is efficient, its complexity is determined. Using decision trees as models, a lower bound for the 
worst-case complexity of sorting algorithms that are based on binary comparisons can be found. 

We can use decision trees to model sorting algorithms and to determine an estimate for the 
worst-case complexity of these algorithms. Note that given n elements, there are n! possible 
orderings of these elements, because each of the n! permutations of these elements can be the 
correct order. The sorting algorithms studied in this book, and most commonly used sorting 
algorithms, are based on binary comparisons, that is, the comparison of two elements at a time. 
The result of each such comparison narrows down the set of possible orderings. Thus, a sorting 
algorithm based on binary comparisons can be represented by a binary decision tree in which 
each internal vertex represents a comparison of two elements. Each leaf represents one of the n! 
permutations of n elements. 
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FIGURE 3 A Decision Tree for Locating a Counterfeit Coin. The counterfeit coin is shown in color 
below each final weighing. 


EXAMPLE 4 We display in Figure 4 a decision tree that orders the elements of the list a, b, c. < 


FIGURE 4 _ A Decision Tree for Sorting Three Distinct Elements. 


The complexity of a sort based on binary comparisons is measured in terms of the number 
of such comparisons used. The largest number of binary comparisons ever needed to sort a list 
with n elements gives the worst-case performance of the algorithm. The most comparisons used 
equals the longest path length in the decision tree representing the sorting procedure. In other 
words, the largest number of comparisons ever needed is equal to the height of the decision 
tree. Because the height of a binary tree with 7! leaves is at least [log n!] (using Corollary | in 
Section 11.1), at least [log n!] comparisons are needed, as stated in Theorem 1. 


THEOREM 1 _ A sorting algorithm based on binary comparisons requires at least [log n!] comparisons. 


We can use Theorem | to provide a big-Omega estimate for the number of comparisons used 
by a sorting algorithm based on binary comparison. We need only note that by Exercise 72 in 
Section 3.2 we know that [logn!] is O(n logn), one of the commonly used reference functions 
for the computational complexity of algorithms. Corollary | is a consequence of this estimate. 
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The number of comparisons used by a sorting algorithm to sort n elements based on binary 
comparisons is Q2(n logn). 


A consequence of Corollary | is that a sorting algorithm based on binary comparisons that 
uses ©(n logn) comparisons, in the worst case, to sort n elements is optimal, in the sense that 
no other such algorithm has better worst-case complexity. Note that by Theorem | in Section 5.4 
we see that the merge sort algorithm is optimal in this sense. 

We can also establish a similar result for the average-case complexity of sorting algorithms. 
The average number of comparisons used by a sorting algorithm based on binary comparisons is 
the average depth of a leaf in the decision tree representing the sorting algorithm. By Exercise 48 
in Section 11.1 we know that the average depth of a leaf in a binary tree with N vertices 
is Q(log NV). We obtain the following estimate when we let N = n! and note that a function that 
is Q(logn!) is also Q (n logn) because log n! is O(n logn). 


The average number of comparisons used by a sorting algorithm to sort n elements based on 
binary comparisons is (2 (n logn). 


Prefix Codes 


Consider the problem of using bit strings to encode the letters of the English alphabet (where 
no distinction is made between lowercase and uppercase letters). We can represent each letter 
with a bit string of length five, because there are only 26 letters and there are 32 bit strings of 
length five. The total number of bits used to encode data is five times the number of characters 
in the text when each character is encoded with five bits. Is it possible to find a coding scheme 
of these letters such that, when data are coded, fewer bits are used? We can save memory and 
reduce transmittal time if this can be done. 

Consider using bit strings of different lengths to encode letters. Letters that occur more 
frequently should be encoded using short bit strings, and longer bit strings should be used to 
encode rarely occurring letters. When letters are encoded using varying numbers of bits, some 
method must be used to determine where the bits for each character start and end. For instance, 
if e were encoded with 0, a with 1, and ¢ with 01, then the bit string 0101 could correspond to 
eat, tea, eaea, OF tt. 

One way to ensure that no bit string corresponds to more than one sequence of letters is 
to encode letters so that the bit string for a letter never occurs as the first part of the bit string 
for another letter. Codes with this property are called prefix codes. For instance, the encoding 
of e as 0, a as 10, and ¢ as 11 is a prefix code. A word can be recovered from the unique bit 
string that encodes its letters. For example, the string 10110 is the encoding of ate. To see this, 
note that the initial 1 does not represent a character, but 10 does represent a (and could not be 
the first part of the bit string of another letter). Then, the next | does not represent a character, 
but 11 does represent ¢. The final bit, 0, represents e. 

A prefix code can be represented using a binary tree, where the characters are the labels of 
the leaves in the tree. The edges of the tree are labeled so that an edge leading to a left child is 
assigned a 0 and an edge leading to a right child is assigned a 1. The bit string used to encode 
a character is the sequence of labels of the edges in the unique path from the root to the leaf 
that has this character as its label. For instance, the tree in Figure 5 represents the encoding of 
e by 0, a by 10, t by 110, 1 by 1110, and s by 1111. 

The tree representing a code can be used to decode a bit string. For instance, consider the 
word encoded by 11111011100 using the code in Figure 5. This bit string can be decoded by 
starting at the root, using the sequence of bits to form a path that stops when a leaf is reached. 


Links 
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Each 0 bit takes the path down the edge leading to the left child of the last vertex in the path, and 
each | bit corresponds to the right child of this vertex. Consequently, the initial 1111 corresponds 
to the path starting at the root, going right four times, leading to a leaf in the graph that has s 
as its label, because the string 1111 is the code for s. Continuing with the fifth bit, we reach a 
leaf next after going right then left, when the vertex labeled with a, which is encoded by 10, is 
visited. Starting with the seventh bit, we reach a leaf next after going right three times and then 
left, when the vertex labeled with n, which is encoded by 1110, is visited. Finally, the last bit, 0, 
leads to the leaf that is labeled with e. Therefore, the original word is sane. 

We can construct a prefix code from any binary tree where the left edge at each internal 
vertex is labeled by 0 and the right edge by a | and where the leaves are labeled by characters. 
Characters are encoded with the bit string constructed using the labels of the edges in the unique 
path from the root to the leaves. 


HUFFMAN CODING We now introduce an algorithm that takes as input the frequencies 
(which are the probabilities of occurrences) of symbols in a string and produces as output a 
prefix code that encodes the string using the fewest possible bits, among all possible binary 
prefix codes for these symbols. This algorithm, known as Huffman coding, was developed by 
David Huffman in a term paper he wrote in 1951 while a graduate student at MIT. (Note that this 
algorithm assumes that we already know how many times each symbol occurs in the string, so we 
can compute the frequency of each symbol by dividing the number of times this symbol occurs 
by the length of the string.) Huffman coding is a fundamental algorithm in data compression, 
the subject devoted to reducing the number of bits required to represent information. Huffman 
coding is extensively used to compress bit strings representing text and it also plays an important 
role in compressing audio and image files. 

Algorithm 2 presents the Huffman coding algorithm. Given symbols and their frequencies, 
our goal is to construct a rooted binary tree where the symbols are the labels of the leaves. The 
algorithm begins with a forest of trees each consisting of one vertex, where each vertex has 
a symbol as its label and where the weight of this vertex equals the frequency of the symbol 
that is its label. At each step, we combine two trees having the least total weight into a single 
tree by introducing a new root and placing the tree with larger weight as its left subtree and the 
tree with smaller weight as its right subtree. Furthermore, we assign the sum of the weights of 
the two subtrees of this tree as the total weight of the tree. (Although procedures for breaking 
ties by choosing between trees with equal weights can be specified, we will not specify such 
procedures here.) The algorithm is finished when it has constructed a tree, that is, when the 
forest is reduced to a single tree. 


DAVID A. HUFFMAN (1925-1999) David Huffman grew up in Ohio. At the age of 18 he received his B.S. 
in electrical engineering from The Ohio State University. Afterward he served in the U.S. Navy as a radar 
maintenance officer on a destroyer that had the mission of clearing mines in Asian waters after World War IL. 
Later, he earned his M.S. from Ohio State and his Ph.D. in electrical engineering from MIT. Huffman joined 
the MIT faculty in 1953, where he remained until 1967 when he became the founding member of the computer 
science department at the University of California at Santa Cruz. He played an important role in developing this 
department and spent the remainder of his career there, retiring in 1994. 

Huffman is noted for his contributions to information theory and coding, signal designs for radar and 
for communications, and design procedures for asynchronous logical circuits. His work on surfaces with zero 


curvature led him to develop original techniques for folding paper and vinyl into unusual shapes considered works of art by many 
and publicly displayed in several exhibits. However, Huffman is best known for his development of what is now called Huffman 
coding, a result of a term paper he wrote during his graduate work at MIT. 

Huffman enjoyed exploring the outdoors, hiking, and traveling extensively. He became certified as a scuba diver when he was 
in his late 60s. He kept poisonous snakes as pets. 
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ALGORITHM 2 Huffman Coding. 


procedure Huffman(C: symbols a; with frequencies w;,i = 1, ...,n) 
F := forest of n rooted trees, each consisting of the single vertex a; and assigned weight w; 
while F is not a tree 
Replace the rooted trees T and T’ of least weights from F with w(T) > w(T’) with a tree 
having a new root that has T as its left subtree and 7’ as its right subtree. Label the new 
edge to T with 0 and the new edge to T’ with 1. 
Assign w(T) + w(T’) as the weight of the new tree. 
{the Huffman coding for the symbol a; is the concatenation of the labels of the edges in the 
unique path from the root to the vertex a; } 


Example 5 illustrates how Algorithm 2 is used to encode a set of five symbols. 


Use Huffman coding to encode the following symbols with the frequencies listed: A: 0.08, B: 
0.10, C: 0.12, D: 0.15, E: 0.20, F: 0.35. What is the average number of bits used to encode a 
character? 


Solution: Figure 6 displays the steps used to encode these symbols. The encoding produced 
encodes A by 111, B by 110, C by 011, D by 010, E by 10, and F by 00. The average number 
of bits used to encode a symbol using this encoding is 


3-0.08+ 3-0.10+3-0.12+3-0.154+2-0.20+2-0.35 = 2.45. < 


Note that Huffman coding is a greedy algorithm. Replacing the two subtrees with the 
smallest weight at each step leads to an optimal code in the sense that no binary prefix code 
for these symbols can encode these symbols using fewer bits. We leave the proof that Huffman 
codes are optimal as Exercise 32. 

There are many variations of Huffman coding. For example, instead of encoding single 
symbols, we can encode blocks of symbols of a specified length, such as blocks of two symbols. 
Doing so may reduce the number of bits required to encode the string (see Exercise 30). We can 
also use more than two symbols to encode the original symbols in the string (see the preamble 
to Exercise 28). Furthermore, a variation known as adaptive Huffman coding (see [Sa00]) can 
be used when the frequency of each symbol in a string is not known in advance, so that encoding 
is done at the same time the string is being read. 


Game Trees 


Trees can be used to analyze certain types of games such as tic-tac-toe, nim, checkers, and chess. 
In each of these games, two players take turns making moves. Each player knows the moves 
made by the other player and no element of chance enters into the game. We model such games 
using game trees; the vertices of these trees represent the positions that a game can be in as it 
progresses; the edges represent legal moves between these positions. Because game trees are 
usually large, we simplify game trees by representing all symmetric positions of a game by the 
same vertex. However, the same position of a game may be represented by different vertices 
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FIGURE 6 Huffman Coding of Symbols in Example 4. 


if different sequences of moves lead to this position. The root represents the starting position. 
The usual convention is to represent vertices at even levels by boxes and vertices at odd levels 
by circles. When the game is in a position represented by a vertex at an even level, it is the first 
player’s move; when the game is in a position represented by a vertex at an odd level, it is the 
second player’s move. Game trees may be infinite when the games they represent never end, 
such as games that can enter infinite loops, but for most games there are rules that lead to finite 
game trees. 

The leaves of a game tree represent the final positions of a game. We assign a value to each 
leaf indicating the payoff to the first player if the game terminates in the position represented 
by this leaf. For games that are win-—lose, we label a terminal vertex represented by a circle with 
a | to indicate a win by the first player and we label a terminal vertex represented by a box with 
a —1 to indicate a win by the second player. For games where draws are allowed, we label a 
terminal vertex corresponding to a draw position with a 0. Note that for win-lose games, we 
have assigned values to terminal vertices so that the larger the value, the better the outcome for 
the first player. 

In Example 6 we display a game tree for a well-known and well-studied game. 
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FIGURE 7 The Game Tree for a Game of Nim. 


Nim _ In a version of the game of nim, at the start of a game there are a number of piles of 
stones. Two players take turns making moves; a legal move consists of removing one or more 
stones from one of the piles, without removing all the stones left. A player without a legal move 
loses. (Another way to look at this is that the player removing the last stone loses because the 
position with no piles of stones is not allowed.) The game tree shown in Figure 7 represents this 
version of nim given the starting position where there are three piles of stones containing two, 
two, and one stone each, respectively. We represent each position with an unordered list of the 
number of stones in the different piles (the order of the piles does not matter). The initial move 
by the first player can lead to three possible positions because this player can remove one stone 
from a pile with two stones (leaving three piles containing one, one, and two stones); two stones 
from a pile containing two stones (leaving two piles containing two stones and one stone); or 
one stone from the pile containing one stone (leaving two piles of two stones). When only one 
pile with one stone is left, no legal moves are possible, so such positions are terminal positions. 
Because nim is a win-lose game, we label the terminal vertices with +1 when they represent 
wins for the first player and —1 when they represent wins for the second player. 


Tic-tac-toe The game tree for tic-tac-toe is extremely large and cannot be drawn here, although 
acomputer could easily build such a tree. We show a portion of the game tic-tac-toe in Figure 8(a). 
Note that by considering symmetric positions equivalent, we need only consider three possible 
initial moves, as shown in Figure 8(a). We also show a subtree of this game tree leading to 
terminal positions in Figure 8(b), where a player who can win makes a winning move. < 


We can recursively define the values of all vertices in a game tree in a way that enables 
us to determine the outcome of this game when both players follow optimal strategies. By a 
strategy we mean a set of rules that tells a player how to select moves to win the game. An 
optimal strategy for the first player is a strategy that maximizes the payoff to this player and for 
the second player is a strategy that minimizes this payoff. We now recursively define the value 
of a vertex. 
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FIGURE 8 Some of the Game Tree for Tic-Tac-Toe. 


DEFINITION 1 ~The value of a vertex in a game tree is defined recursively as: 


(i) the value of a leaf is the payoff to the first player when the game terminates in the 
position represented by this leaf. 


(ii) the value of an internal vertex at an even level is the maximum of the values of its 
children, and the value of an internal vertex at an odd level is the minimum of the 
values of its children. 


The strategy where the first player moves to a position represented by a child with maximum 
value and the second player moves to a position of a child with minimum value is called the 
minmax strategy. We can determine who will win the game when both players follow the 
minmax strategy by calculating the value of the root of the tree; this value is called the value of 
the tree. This is a consequence of Theorem 3. 


THEOREM 3 _ The value of a vertex of a game tree tells us the payoff to the first player if both players follow 
the minmax strategy and play starts from the position represented by this vertex. 
Proof: We will use induction to prove this theorem. 


BASIS STEP: If the vertex is a leaf, by definition the value assigned to this vertex is the payoff 
to the first player. 
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FIGURE 9 Showing the Values of Vertices in the Game of Nim. 


INDUCTIVE STEP: The inductive hypothesis is the assumption that the values of the children 
of a vertex are the payoffs to the first player, assuming that play starts at each of the positions 
represented by these vertices. We need to consider two cases, when it is the first player’s turn 
and when it is the second player’s turn. 

When it is the first player’s turn, this player follows the minmax strategy and moves to the 
position represented by the child with the largest value. By the inductive hypothesis, this value 
is the payoff to the first player when play starts at the position represented by this child and 
follows the minmax strategy. By the recursive step in the definition of the value of an internal 
vertex at an even level (as the maximum value of its children), the value of this vertex is the 
payoff when play begins at the position represented by this vertex. 

When it is the second player’s turn, this player follows the minmax strategy and moves to 
the position represented by the child with the least value. By the inductive hypothesis, this value 
is the payoff to the first player when play starts at the position represented by this child and 
both players follow the minmax strategy. By the recursive definition of the value of an internal 
vertex at an odd level as the minimum value of its children, the value of this vertex is the payoff 
when play begins at the position represented by this vertex. 


Remark: By extending the proof of Theorem 3, it can be shown that the minmax strategy is the 
optimal strategy for both players. 


Example 8 illustrates how the minmax procedure works. It displays the values assigned to 
the internal vertices in the game tree from Example 6. Note that we can shorten the computation 
required by noting that for win—lose games, once a child of a square vertex with value +1 
is found, the value of the square vertex is also +1 because +1 is the largest possible payoff. 
Similarly, once a child of a circle vertex with value —1 is found, this is the value of the circle 
vertex also. 


In Example 6 we constructed the game tree for nim with a starting position where there are 
three piles containing two, two, and one stones. In Figure 9 we show the values of the vertices 
of this game tree. The values of the vertices are computed using the values of the leaves and 
working one level up at a time. In the right margin of this figure we indicate whether we use 
the maximum or minimum of the values of the children to find the value of an internal vertex at 
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each level. For example, once we have found the values of the three children of the root, which 
are 1, —1, and —1, we find the value of the root by computing max(1, —1, —1) = 1. Because 
the value of the root is 1, it follows that the first player wins when both players follow a minmax 
strategy. < 


Game trees for some well-known games can be extraordinarily large, because these games 
have many different possible moves. For example, the game tree for chess has been estimated to 
have as many as 10!°° vertices! It may be impossible to use Theorem 3 directly to study a game 
because of the size of the game tree. Therefore, various approaches have been devised to help 
determine good strategies and to determine the outcome of such games. One useful technique, 
called alpha-beta pruning, eliminates much computation by pruning portions of the game tree 
that cannot affect the values of ancestor vertices. (For information about alpha-beta pruning, 
consult [Gr90].) Another useful approach is to use evaluation functions, which estimate the value 
of internal vertices in the game tree when it is not feasible to compute these values exactly. For 
example, in the game of tic-tac-toe, as an evaluation function for a position, we may use the 
number of files (rows, columns, and diagonals) containing no Os (used to indicate moves of the 
second player) minus the number of files containing no Xs (used to indicate moves of the first 
player). This evaluation function provides some indication of which player has the advantage in 
the game. Once the values of an evaluation function are inserted, the value of the game can be 
computed following the rules used for the minmax strategy. Computer programs created to play 
chess, such as the famous Deep Blue program, are based on sophisticated evaluation functions. 
For more information about how computers play chess see [Le91]. 


1. Build a binary search tree for the words banana, peach, 
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elements and devise an algorithm that sorts these elements 
using this number of comparisons. 
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The tournament sort is a sorting algorithm that works by 
building an ordered binary tree. We represent the elements to 
be sorted by vertices that will become the leaves. We build up 
the tree one level at a time as we would construct the tree repre- 
senting the winners of matches in a tournament. Working left 
to right, we compare pairs of consecutive elements, adding a 
parent vertex labeled with the larger of the two elements under 
comparison. We make similar comparisons between labels of 
vertices at each level until we reach the root of the tree that is 
labeled with the largest element. The tree constructed by the 
tournament sort of 22, 8, 14, 17, 3, 9, 27, 11 is illustrated in 
part (a) of the figure. Once the largest element has been de- 
termined, the leaf with this label is relabeled by —oo, which 
is defined to be less than every element. The labels of all ver- 
tices on the path from this vertex up to the root of the tree are 
recalculated, as shown in part (b) of the figure. This produces 
the second largest element. This process continues until the 
entire list has been sorted. 


13. Complete the tournament sort of the list 22, 8, 14, 17, 3, 
9, 27, 11. Show the labels of the vertices at each step. 


14. Use the tournament sort to sort the list 17, 4, 1, 5, 13, 10, 
14, 6. 


15. Describe the tournament sort using pseudocode. 


16. Assuming that n, the number of elements to be sorted, 
equals 2* for some positive integer k, determine the num- 
ber of comparisons used by the tournament sort to find 
the largest element of the list using the tournament sort. 


17. How many comparisons does the tournament sort use to 
find the second largest, the third largest, and so on, up to 
the (n — 1)st largest (or second smallest) element? 


18. Show that the tournament sort requires O(n logn) com- 
parisons to sort a list of n elements. [Hint: By inserting 
the appropriate number of dummy elements defined to be 
smaller than all integers, such as —oo, assume that n = ok 
for some positive integer k.] 


19. Which of these codes are prefix codes? 
a) a: 11, e: 00, t: 10, 5: 01 
b) a: 0, e: 1, t: 01, s: 001 
c) a: 101, e: 11, t: 001, s: 011, n: 010 
d) a: 010, e: 11, t: 011, s: 1011, n: 1001, 7: 10101 


20. Construct the binary tree with prefix codes representing 
these coding schemes. 


a) a: 11, e:0,t: 101, 5: 100 
b) a: 1, e: 01, t: 001, s: 0001, n: 00001 
c) a: 1010, e: 0, t: 11, s: 1011, : 1001, i: 10001 


21. What are the codes for a, e, i, k, o, p, and u if the coding 
scheme is represented by this tree? 


22. Given the coding scheme a: 001, b: 0001, e: 1, r: 0000, 
s: 0100, t: 011, x: 01010, find the word represented by 


a) 01110100011. b) 0001110000. 
c) 0100101010. d) 01100101010. 


23. Use Huffman coding to encode these symbols with given 
frequencies: a: 0.20, b: 0.10, c: 0.15, d: 0.25, e: 0.30. 
What is the average number of bits required to encode a 
character? 


24. Use Huffman coding to encode these symbols with given 
frequencies: A: 0.10, B: 0.25, C: 0.05, D: 0.15, E: 0.30, 
F: 0.07, G: 0.08. What is the average number of bits re- 
quired to encode a symbol? 


25. Construct two different Huffman codes for these symbols 
and frequencies: t: 0.2, u: 0.3, v: 0.2, w: 0.3. 


26. a) Use Huffman coding to encode these symbols with 
frequencies a: 0.4, b: 0.2, c: 0.2, d: 0.1, e: 0.1 in two 
different ways by breaking ties in the algorithm dif- 
ferently. First, among the trees of minimum weight 
select two trees with the largest number of vertices 
to combine at each stage of the algorithm. Second, 
among the trees of minimum weight select two trees 
with the smallest number of vertices at each stage. 


b) Compute the average number of bits required to en- 
code a symbol with each code and compute the vari- 
ances of this number of bits for each code. Which 
tie-breaking procedure produced the smaller variance 
in the number of bits required to encode a symbol? 


27. 


Construct a Huffman code for the letters of the English al- 
phabet where the frequencies of letters in typical English 
text are as shown in this table. 


Pa 
A N 


B 
C 
D 
E 
F 
G 
H 
I 
J 
K 
L 
M 


NK eK Ed CHMNROWO 


Suppose that m is a positive integer with m > 2. An m-ary 
Huffman code for a set of N symbols can be constructed anal- 
ogously to the construction of a binary Huffman code. At the 
initial step, ((V — 1) mod (m — 1)) + | trees consisting of a 
single vertex with least weights are combined into a rooted 
tree with these vertices as leaves. At each subsequent step, 
the m trees of least weight are combined into an m-ary tree. 


28. 


29. 


30. 


31. 


* 32. 


Describe the m-ary Huffman coding algorithm in 
pseudocode. 


Using the symbols 0, 1, and 2 use ternary (m = 3) 
Huffman coding to encode these letters with the given 
frequencies: A: 0.25, E: 0.30, N: 0.10, R: 0.05, T: 0.12, 
Z: 0.18. 


Consider the three symbols A, B, and C with frequencies 

A: 0.80, B: 0.19, C: 0.01. 

a) Construct a Huffman code for these three symbols. 

b) Form a new set of nine symbols by grouping together 
blocks of two symbols, AA, AB, AC, BA, BB, BC, 
CA, CB, and CC. Construct a Huffman code for these 
nine symbols, assuming that the occurrences of sym- 
bols in the original text are independent. 

c) Compare the average number of bits required to en- 
code text using the Huffman code for the three sym- 
bols in part (a) and the Huffman code for the nine 
blocks of two symbols constructed in part (b). Which 
is more efficient? 

Given n + 1 symbols x1, x2,...,%Xn,Xn+1 appearing 1, 

Si; fo,---5 fn times in a symbol string, respectively, 

where fj is the jth Fibonacci number, what is the max- 

imum number of bits used to encode a symbol when all 
possible tie-breaking selections are considered at each 
stage of the Huffman coding algorithm? 


Show that Huffman codes are optimal in the sense that 
they represent a string of symbols using the fewest bits 
among all binary prefix codes. 


33. 


34. 


35. 


36. 


37. 


38. 
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Draw a game tree for nim if the starting position consists 
of two piles with two and three stones, respectively. When 
drawing the tree represent by the same vertex symmetric 
positions that result from the same move. Find the value 
of each vertex of the game tree. Who wins the game if 
both players follow an optimal strategy? 


Draw a game tree for nim if the starting position consists 
of three piles with one, two, and three stones, respectively. 
When drawing the tree represent by the same vertex sym- 
metric positions that result from the same move. Find the 
value of each vertex of the game tree. Who wins the game 
if both players follow an optimal strategy? 


Suppose that we vary the payoff to the winning player 
in the game of nim so that the payoff is n dollars when 
n is the number of legal moves made before a terminal 
position is reached. Find the payoff to the first player if 
the initial position consists of 

a) two piles with one and three stones, respectively. 

b) two piles with two and four stones, respectively. 


c) three piles with one, two, and three stones, 
respectively. 


Suppose that in a variation of the game of nim we allow a 
player to either remove one or more stones from a pile or 
merge the stones from two piles into one pile as long as at 
least one stone remains. Draw the game tree for this vari- 
ation of nim if the starting position consists of three piles 
containing two, two, and one stone, respectively. Find the 
values of each vertex in the game tree and determine the 
winner if both players follow an optimal strategy. 


Draw the subtree of the game tree for tic-tac-toe begin- 
ning at each of these positions. Determine the value of 
each of these subtrees. 


a) O| X| x b) x |O| x 
xX }O}O OO} X/|X 
X O 
) x| jo d) O| xX 
olo]| — x|o 
x Xx X|O 
Suppose that the first four moves of a tic-tac-toe game are 


as shown. Does the first player (whose moves are marked 
by Xs) have a strategy that will always win? 


a) x1O|xX b) x/O|xX 
a, 
=e 
c) O| xX d) x 
x] x lo 
o| | | O 
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39. 


40. 


41. 


42. 
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Show that if a game of nim begins with two piles contain- 
ing the same number of stones, as long as this number is at 
least two, then the second player wins when both players 
follow optimal strategies. 


Show that if a game of nim begins with two piles con- 
taining different numbers of stones, the first player wins 
when both players follow optimal strategies. 


How many children does the root of the game tree for 
checkers have? How many grandchildren does it have? 


How many children does the root of the game tree for 
nim have and how many grandchildren does it have if the 
starting position is 

a) piles with four and five stones, respectively. 

b) piles with two, three, and four stones, respectively. 


Tree Traversal 


43. 


44, 


c) piles with one, two, three, and four stones, respec- 
tively. 
d) piles with two, two, three, three, and five stones, re- 
spectively. 
Draw the game tree for the game of tic-tac-toe for the lev- 
els corresponding to the first two moves. Assign the value 
of the evaluation function mentioned in the text that as- 
signs to a position the number of files containing no Os 
minus the number of files containing no Xs as the value of 
each vertex at this level and compute the value of the tree 
for vertices as if the evaluation function gave the correct 
values for these vertices. 
Use pseudocode to describe an algorithm for determin- 
ing the value of a game tree when both players follow a 
minmax strategy. 


Introduction 


Links 


Ordered rooted trees are often used to store information. We need procedures for visiting each 
vertex of an ordered rooted tree to access data. We will describe several important algorithms 


for visiting all the vertices of an ordered rooted tree. Ordered rooted trees can also be used 
to represent various types of expressions, such as arithmetic expressions involving numbers, 
variables, and operations. The different listings of the vertices of ordered rooted trees used to 
represent expressions are useful in the evaluation of these expressions. 


Universal Address Systems 


Procedures for traversing all vertices of an ordered rooted tree rely on the orderings of children. 
In ordered rooted trees, the children of an internal vertex are shown from left to right in the 
drawings representing these directed graphs. 

We will describe one way we can totally order the vertices of an ordered rooted tree. To 
produce this ordering, we must first label all the vertices. We do this recursively: 


1. Label the root with the integer 0. Then label its k children (at level 1) from left to right 


with 1,2,3,...,k. 


2. For each vertex v at level n with label A, label its k, children, as they are drawn from left 
to right, with A.1, A.2,..., A.ky. 


Following this procedure, a vertex v at level n, for n > 1, is labeled x;.x2..... Xn, where the 
unique path from the root to v goes through the x; st vertex at level 1, the xznd vertex at level 2, 
and so on. This labeling is called the universal address system of the ordered rooted tree. 

We can totally order the vertices using the lexicographic ordering of their labels in the univer- 


sal address system. The vertex labeled x1 .x2 
if there is ani,0 <i <n, with x; = yj, x2 = yo,.. 


and x; = yj fori = 1,2,...,n. 


fe Xy is less than the vertex labeled y,.y2.....¥m 
., Xj-1 = Yi-1, and x; < yj; orifn <m 


EXAMPLE 1 


Extra 
Examples 


DEFINITION 1 


EXAMPLE 2 


Extra 
Examples 


~@ 


i 
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3.1.2.1 | 3 23 | 
3,122 3124 


FIGURE 1 The Universal Address System of an Ordered Rooted Tree. 


We display the labelings of the universal address system next to the vertices in the ordered rooted 
tree shown in Figure |. The lexicographic ordering of the labelings is 


0<1<11<12<13<2<3<3.1 <3.1.1 < 3.1.2 < 3.1.2.1 < 3.1.2.2 < 
< 3.1.2.3 <3.1.2.4<3.1.3<32<4<41<5<51 <5.1.1<52<5.3 


Traversal Algorithms 


Procedures for systematically visiting every vertex of an ordered rooted tree are called traversal 
algorithms. We will describe three of the most commonly used such algorithms, preorder 
traversal, inorder traversal, and postorder traversal. Each of these algorithms can be defined 
recursively. We first define preorder traversal. 


Let T be an ordered rooted tree with root r. If T consists only of r, then r is the preorder 
traversal of T . Otherwise, suppose that 7), 72, ..., T, are the subtrees at r from left to right 
in T. The preorder traversal begins by visiting r. It continues by traversing 7 in preorder, 
then 7> in preorder, and so on, until 7; is traversed in preorder. 


The reader should verify that the preorder traversal of an ordered rooted tree gives the same 
ordering of the vertices as the ordering obtained using a universal address system. Figure 2 
indicates how a preorder traversal is carried out. 

Example 2 illustrates preorder traversal. 


In which order does a preorder traversal visit the vertices in the ordered rooted tree T shown in 
Figure 3? 


Solution: The steps of the preorder traversal of T are shown in Figure 4. We traverse T in 
preorder by first listing the root a, followed by the preorder list of the subtree with root b, the 
preorder list of the subtree with root c (which is just c) and the preorder list of the subtree with 
root d. 
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Step 1: Visit r 


Preorder traversal 


Step 2: Step 3: Stepn+1: 
Visit T, Visit T> Visit T,, 
in preorder in preorder in preorder 
FIGURE 2 Preorder Traversal. FIGURE 3 The Ordered 
Rooted Tree T. 
a 


Preorder traversal: Visit root, 
visit subtrees left to right 


FIGURE 4 _ The Preorder Traversal of T. 


DEFINITION 2 


EXAMPLE 3 


Extra p> 
Examples 2 
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Step 2: Visit r 


Inorder traversal 


Step 1: Step 3: Stepn+1: 
Visit 7, in Visit T> in Visit 7, in 
inorder inorder inorder 


FIGURE 5 _ Inorder Traversal. 


The preorder list of the subtree with root b begins by listing b, then the vertices of the 
subtree with root e in preorder, and then the subtree with root f in preorder (which is just f). 
The preorder list of the subtree with root d begins by listing d, followed by the preorder list of 
the subtree with root g, followed by the subtree with root h (which is just h), followed by the 
subtree with root i (which is just 7). 

The preorder list of the subtree with root e begins by listing e, followed by the preorder 
listing of the subtree with root j (which is just 7), followed by the preorder listing of the subtree 
with root k. The preorder listing of the subtree with root g is g followed by /, followed by m. 
The preorder listing of the subtree with root k is k,n, o, p. Consequently, the preorder traversal 
of T isa, b, e, j,k, n, 0, p, f,c, d, g, l,m, h, i. < 


We will now define inorder traversal. 


Let T be an ordered rooted tree with root r. If T consists only of r, then r is the inorder 


traversal of T. Otherwise, suppose that 7,, T>, ..., 7, are the subtrees at r from left to right. 
The inorder traversal begins by traversing 7; in inorder, then visiting r. It continues by 
traversing T> in inorder, then 73 in inorder, ..., and finally T,, in inorder. 


Figure 5 indicates how inorder traversal is carried out. Example 3 illustrates how inorder 
traversal is carried out for a particular tree. 


In which order does an inorder traversal visit the vertices of the ordered rooted tree T in Figure 3? 


Solution: The steps of the inorder traversal of the ordered rooted tree T are shown in Figure 6. 
The inorder traversal begins with an inorder traversal of the subtree with root b, the root a, the 
inorder listing of the subtree with root c, which is just c, and the inorder listing of the subtree 
with root d. 

The inorder listing of the subtree with root b begins with the inorder listing of the subtree 
with root e, the root b, and f. The inorder listing of the subtree with root d begins with the 
inorder listing of the subtree with root g, followed by the root d, followed by h, followed by i. 

The inorder listing of the subtree with root e is 7, followed by the root e, followed by the 
inorder listing of the subtree with root k. The inorder listing of the subtree with root g is /, g,m. 
The inorder listing of the subtree with root k is n, k, o, p. Consequently, the inorder listing of 
the ordered rooted tree is j, e,n, k, 0, p, b, f,a,c,l, g,m,d,h,i. 
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Inorder traversal: Visit leftmost 
subtree, visit root, visit other 
subtrees left to right 


m di ih i 


FIGURE 6 The Inorder Traversal of T. 


We now define postorder traversal. 


DEFINITION 3 Let T be an ordered rooted tree with root r. If T consists only of r, then r is the postorder 
traversal of T. Otherwise, suppose that 7), 7>,..., T, are the subtrees at r from left to 
right. The postorder traversal begins by traversing T, in postorder, then 7> in postorder, ..., 
then 7, in postorder, and ends by visiting r. 


Figure 7 illustrates how postorder traversal is done. Example 4 illustrates how postorder 
traversal works. 


EXAMPLE 4 


Extra pe 
Examples 2 
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Step n+ 1: Visit r 


Postorder traversal 


Step 1: Step 2: Step n: 
Visit T; Visit T> Visit T,, 
in postorder in postorder in postorder 


FIGURE 7 Postorder Traversal. 


In which order does a postorder traversal visit the vertices of the ordered rooted tree T shown 
in Figure 3? 


Solution: The steps of the postorder traversal of the ordered rooted tree T are shown in Figure 8. 
The postorder traversal begins with the postorder traversal of the subtree with root b, the postorder 
traversal of the subtree with root c, which is just c, the postorder traversal of the subtree with 
root d, followed by the root a. 

The postorder traversal of the subtree with root b begins with the postorder traversal of the 
subtree with root e, followed by f, followed by the root b. The postorder traversal of the rooted 
tree with root d begins with the postorder traversal of the subtree with root g, followed by h, 
followed by 7, followed by the root d. 

The postorder traversal of the subtree with root e begins with j, followed by the postorder 
traversal of the subtree with root k, followed by the root e. The postorder traversal of the subtree 
with root g is /, m, g. The postorder traversal of the subtree with root k is n, 0, p, k. Therefore, 
the postorder traversal of T is j,n, 0, p, k, e, f, b,c, 1,m, g,h,i,d,a. < 


There are easy ways to list the vertices of an ordered rooted tree in preorder, inorder, and 
postorder. To do this, first draw a curve around the ordered rooted tree starting at the root, moving 
along the edges, as shown in the example in Figure 9. We can list the vertices in preorder by 
listing each vertex the first time this curve passes it. We can list the vertices in inorder by listing 
a leaf the first time the curve passes it and listing each internal vertex the second time the curve 
passes it. We can list the vertices in postorder by listing a vertex the last time it is passed on the 
way back up to its parent. When this is done in the rooted tree in Figure 9, it follows that the 
preorder traversal gives a, b,d,h,e,i, j, c, f, g, k, the inorder traversal gives h, d, b, i, e, j, 
a, f,c, k, g; and the postorder traversal gives h, d,i, j,e, b, f, k, g,c, a. 

Algorithms for traversing ordered rooted trees in preorder, inorder, or postorder are most 
easily expressed recursively. 


ALGORITHM 1 Preorder Traversal. 


procedure preorder(T: ordered rooted tree) 
r := root of T 
list r 
for each child c of r from left to right 
T (c) := subtree with c as its root 
preorder(T (c)) 
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Postorder traversal: Visit 
subtrees left to right; visit root 


e 
h i doa 
e e 
J k e f b ec Ll m gh i dea 
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n oo p 
Jj no p k e f b c lm geih i 
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FIGURE 8 The Postorder Traversal of T. 


FIGURE 9 A Shortcut for Traversing an Ordered 
Rooted Tree in Preorder, Inorder, and Postorder. 


EXAMPLE 5 
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ALGORITHM 2 Inorder Traversal. 


procedure inorder(T: ordered rooted tree) 
r := root of T 
if r is a leaf then list r 
else 
1 := first child of r from left to right 
T (1) := subtree with / as its root 
inorder(T (1)) 
list r 
for each child c of r except for / from left to right 
T (c) := subtree with c as its root 
inorder(T (c)) 


ALGORITHM 3 Postorder Traversal. 


procedure postorder(T: ordered rooted tree) 
r := root of T 
for each child c of r from left to right 
T(c) := subtree with c as its root 
postorder(T (c)) 
list r 


Note that both the preorder traversal and the postorder traversal encode the structure of 
an ordered rooted tree when the number of children of each vertex is specified. That is, an 
ordered rooted tree is uniquely determined when we specify a list of vertices generated by a 
preorder traversal or by a postorder traversal of the tree, together with the number of children 
of each vertex (see Exercises 26 and 27). In particular, both a preorder traversal and a postorder 
traversal encode the structure of a full ordered m-ary tree. However, when the number of children 
of vertices is not specified, neither a preorder traversal nor a postorder traversal encodes the 
structure of an ordered rooted tree (see Exercises 28 and 29). 


Infix, Prefix, and Postfix Notation 


We can represent complicated expressions, such as compound propositions, combinations of 
sets, and arithmetic expressions using ordered rooted trees. For instance, consider the repre- 
sentation of an arithmetic expression involving the operators + (addition), — (subtraction), 
(multiplication), / (division), and * (exponentiation). We will use parentheses to indicate the 
order of the operations. An ordered rooted tree can be used to represent such expressions, where 
the internal vertices represent operations, and the leaves represent the variables or numbers. 
Each operation operates on its left and right subtrees (in that order). 


What is the ordered rooted tree that represents the expression ((x + y) ¢ 2) + ((x — 4)/3)? 


Solution: The binary tree for this expression can be built from the bottom up. First, a sub- 
tree for the expression x + y is constructed. Then this is incorporated as part of the larger 
subtree representing (x + y) +2. Also, a subtree for x — 4 is constructed, and then this is incor- 
porated into a subtree representing (x — 4)/3. Finally the subtrees representing (x + y) t 2 
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EXAMPLE 6 


/\ | /\ /~ 
: : /\ /\ 
AN z \ 7 \ 7 ‘ : 


FIGURE 10 A Binary Tree Representing ((x + y) ¢ 2) + (x — 4)/3). 


and (x — 4)/3 are combined to form the ordered rooted tree representing ((x + y) ¢ 2) + 
((x — 4)/3). These steps are shown in Figure 10. < 


An inorder traversal of the binary tree representing an expression produces the original 
expression with the elements and operations in the same order as they originally occurred, except 
for unary operations, which instead immediately follow their operands. For instance, inorder 
traversals of the binary trees in Figure 11, which represent the expressions (x + y)/(x + 3), 
(x + (y/x)) + 3, and x + (v/(x + 3)), all lead to the infix expression x + y/x + 3. To make 
such expressions unambiguous it is necessary to include parentheses in the inorder traversal 
whenever we encounter an operation. The fully parenthesized expression obtained in this way 
is said to be in infix form. 

We obtain the prefix form of an expression when we traverse its rooted tree in preorder. 
Expressions written in prefix form are said to be in Polish notation, which is named after the 
Polish logician Jan Lukasiewicz. An expression in prefix notation (where each operation has 
a specified number of operands), is unambiguous, so no parentheses are needed in such an 
expression. The verification of this is left as an exercise for the reader. 


What is the prefix form for ((x + y) ¢ 2) + (x — 4)/3)? 


Solution: We obtain the prefix form for this expression by traversing the binary tree that represents 
it in preorder, shown in Figure 10. This produces + + + x y2/— x 43. 


In the prefix form of an expression, a binary operator, such as +, precedes its two operands. 
Hence, we can evaluate an expression in prefix form by working from right to left. When 
we encounter an operator, we perform the corresponding operation with the two operands 


| es 7% 
~~ Ff, /\ 
I, £ FS /\ 


FIGURE 11 Rooted Trees Representing (x + y)/(x +3), (x + (y/x)) +3, and 
x + (y/(x + 3)). 


EXAMPLE 7 


Links 


Reverse polish notation 
was first proposed in 1954 
by Burks, Warren, and 


Wright. 


EXAMPLE 8 


EXAMPLE 9 
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+ -— * 2 3 5 / Ft 2 3 4 2.3 2 = a Pf OS - 
243=8 2*3=6 
+ - * 2 3 5 / 8 4 7 6 = 4 ® 2 f 4 
8/4=2 1-621 
+ - * 2 3 5 2 i A 7. eo Ss sf 
2*3=6 14 =1 
+ - 6 5 2 a ee ae 
6-5=1 9/3 =3 
+ 1 2 a se 
L+2=3 1+3=4 
Value of expression: 3 Value of expression: 4 
FIGURE 12 Evaluating a Prefix FIGURE 13 Evaluating a Postfix 
Expression. Expression. 


immediately to the right of this operand. Also, whenever an operation is performed, we consider 
the result a new operand. 


What is the value of the prefix expression + — * 235/234? 


Solution: The steps used to evaluate this expression by working right to left, and performing 
operations using the operands on the right, are shown in Figure 12. The value of this expression 


is 3. < 


We obtain the postfix form of an expression by traversing its binary tree in postorder. 
Expressions written in postfix form are said to be in reverse Polish notation. Expressions in 
reverse Polish notation are unambiguous, so parentheses are not needed. The verification of this 
is left to the reader. Reverse polish notation was extensively used in electronic calculators in 
the 1970s and 1980s. 


What is the postfix form of the expression ((x + y) ¢ 2) + ((x — 4)/3)? 


Solution: The postfix form of the expression is obtained by carrying out a postorder traversal of 
the binary tree for this expression, shown in Figure 10. This produces the postfix expression: x 
y+2+x4-3/4. 


In the postfix form of an expression, a binary operator follows its two operands. So, to 
evaluate an expression from its postfix form, work from left to right, carrying out operations 
whenever an operator follows two operands. After an operation is carried out, the result of this 
operation becomes a new operand. 


What is the value of the postfix expression 7 2 3 * — 4 + 93/+? 


Solution: The steps used to evaluate this expression by starting at the left and carrying out 
operations when two operands are followed by an operator are shown in Figure 13. The value 
of this expression is 4. < 
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FIGURE 14 Constructing the Rooted Tree for a Compound Proposition. 


Rooted trees can be used to represent other types of expressions, such as those representing 
compound propositions and combinations of sets. In these examples unary operators, such as 
the negation of a proposition, occur. To represent such operators and their operands, a vertex 
representing the operator and a child of this vertex representing the operand are used. 


EXAMPLE 10 _ Find the ordered rooted tree representing the compound proposition (=(p A q)) <= (=p V 7q). 
Then use this rooted tree to find the prefix, postfix, and infix forms of this expression. 


Extra * Solution: The rooted tree for this compound proposition is constructed from the bottom up. First, 
Examples subtrees for =p and —q are formed (where — is considered a unary operator). Also, a subtree 
for p Aq is formed. Then subtrees for —(p A qg) and (—p) V (—q) are constructed. Finally, 
these two subtrees are used to form the final rooted tree. The steps of this procedure are shown 
in Figure 14. 
The prefix, postfix, and infix forms of this expression are found by traversing this rooted tree 
in preorder, postorder, and inorder (including parentheses), respectively. These traversals give 
<> A pq V =p~q, Pd \ =p-qr V <, and (-(p A q)) (Ap) V (-q)), respectively. 4 


Because prefix and postfix expressions are unambiguous and because they can be evaluated 
easily without scanning back and forth, they are used extensively in computer science. Such 
expressions are especially useful in the construction of compilers. 


g JAN LUKASIEWICZ (1878-1956) Jan Lukasiewicz was born into a Polish-speaking family in Lvov. At that 
time Lvov was part of Austria, but it is now in the Ukraine. His father was a captain in the Austrian army. 
Lukasiewicz became interested in mathematics while in high school. He studied mathematics and philosophy 
at the University of Lvov at both the undergraduate and graduate levels. After completing his doctoral work 
he became a lecturer there, and in 1911 he was appointed to a professorship. When the University of Warsaw 
was reopened as a Polish university in 1915, Lukasiewicz accepted an invitation to join the faculty. In 1919 he 
served as the Polish Minister of Education. He returned to the position of professor at Warsaw University where 
he remained from 1920 to 1939, serving as rector of the university twice. 

Lukasiewicz was one of the cofounders of the famous Warsaw School of Logic. He published his famous 
text, Elements of Mathematical Logic, in 1928. With his influence, mathematical logic was made a required course for mathematics 
and science undergraduates in Poland. His lectures were considered excellent, even attracting students of the humanities. 

Lukasiewicz and his wife experienced great suffering during World War II, which he documented in a posthumously published 
autobiography. After the war they lived in exile in Belgium. Fortunately, in 1949 he was offered a position at the Royal Irish Academy 
in Dublin. 

Lukasiewicz worked on mathematical logic throughout his career. His work on a three-valued logic was an important contribution 
to the subject. Nevertheless, he is best known in the mathematical and computer science communities for his introduction of 
parenthesis-free notation, now called Polish notation. 
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Exercises 
In Exercises 1-3 construct the universal address system for the b) 1.1, 1.2.1, 1.2.2, 1.2.3, 2.1, 2.2.1, 2.3.1, 2.3.2, 
given ordered rooted tree. Then use this to order its vertices 2.4.2.1, 2.4.2.2, 3.1, 3.2.1, 3.2.2 
using the lexicographic order of their labels. ce) 1.1, 1.2.1, 1.2.2, 1.2.2.1, 1.3, 1.4, 2, 3.1, 3.2, 4.1.1.1 
1 In Exercises 7-9 determine the order in which a preorder 
° traversal visits the vertices of the given ordered rooted tree. 
7. 
2. 
8. 
3. 


4. Suppose that the address of the vertex v in the ordered 
rooted tree T is 3.4.5.2.4. 


a) At what level is v? 
b) What is the address of the parent of v? 


: doh 5 r 
¢) What is the least number of siblings v can have! 10. In which order are the vertices of the ordered rooted tree 


d) What is the smallest possible number of vertices in T in Exercise 7 visited using an inorder traversal? 

: ; 9 ; : 

if v has this address? 11. In which order are the vertices of the ordered rooted tree 
e) Find the other addresses that must occur. in Exercise 8 visited using an inorder traversal? 

5. Suppose that the vertex with the largest address in an or- 12. In which order are the vertices of the ordered rooted tree 
dered rooted tree T has address 2.3.4.3.1. Is it possible to in Exercise 9 visited using an inorder traversal? 
determine the number of vertices in T? 13. In which order are the vertices of the ordered rooted tree 

6. Can the leaves of an ordered rooted tree have the follow- in Exercise 7 visited using a postorder traversal? 
ing list of universal addresses? If so, construct such an 14. In which order are the vertices of the ordered rooted tree 
ordered rooted tree. in Exercise 8 visited using a postorder traversal? 

a) 1.1.1, 1.1.2, 1.2, 2.1.1.1, 2.1.2, 2.1.3, 2.2, 3.1.1, 15. In which order are the vertices of the ordered rooted tree 


3.1.2.1, 3.1.2.2, 3.2 in Exercise 9 visited using a postorder traversal? 
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16. 


17. 


18. 


19. 


*20. 


*21. 


22. 


23. 


24, 


25. 


* 26. 


* 27, 
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a) Represent the expression 
(y —(3+~x)) — 5 using a binary tree. 
Write this expression in 


((% +2) f 3) * 


b) prefix notation. 

c) postfix notation. 

d) infix notation. 

a) Represent the expressions (x + xy) +(x/y) and 
x + ((xy + x)/y) using binary trees. 

Write these expressions in 

b) prefix notation. 

c) postfix notation. 

d) infix notation. 

a) Represent the compound propositions —=(p A q) = 
(=p V 7q) and (~p Aq < —p)) V 7q using or- 
dered rooted trees. 

Write these expressions in 

b) prefix notation. 

c) postfix notation. 

d) infix notation. 

a) Represent (AN B) — (AU (B-—A)) using an or- 
dered rooted tree. 

Write this expression in 

b) prefix notation. 

c) postfix notation. 

d) infix notation. 

In how many ways can the string ~p Aq < =p V -=q 

be fully parenthesized to yield an infix expression? 

In how many ways can the string AN B— AM B — Abe 

fully parenthesized to yield an infix expression? 

Draw the ordered rooted tree corresponding to each of 

these arithmetic expressions written in prefix notation. 

Then write each expression using infix notation. 

a) +*+-—53214 

b) ¢+23-51 

ec) */93+*24—-—76 

What is the value of each of these prefix expressions? 

a) —*2/843 

b) t}-—*33*425 

ec) +-—432423/6-42 

d) *+34+3%734333 

What is the value of each of these postfix expressions? 

a) 521-—-314++4+*x 

b) 93/54+72-—x 

ce) 32*2¢53-84/x«—- 

Construct the ordered rooted tree whose preorder traver- 

salis a, b, f, c, g, h, i, d, e,j, k, l, where a has four children, 

c has three children, j has two children, b and e have one 

child each, and all other vertices are leaves. 

Show that an ordered rooted tree is uniquely determined 

when a list of vertices generated by a preorder traversal 

of the tree and the number of children of each vertex are 
specified. 

Show that an ordered rooted tree is uniquely determined 

when a list of vertices generated by a postorder traversal 

of the tree and the number of children of each vertex are 
specified. 


28. 


29. 


c df eg h cod 


Show that preorder traversals of the two ordered rooted 
trees displayed below produce the same list of vertices. 
Note that this does not contradict the statement in Ex- 
ercise 26, because the numbers of children of internal 
vertices in the two ordered rooted trees differ. 


a a 


g h 


Show that postorder traversals of these two ordered rooted 
trees produce the same list of vertices. Note that this does 
not contradict the statement in Exercise 27, because the 
numbers of children of internal vertices in the two ordered 
rooted trees differ. 


a a 


bg h 


Well-formed formulae in prefix notation over a set of sym- 
bols and a set of binary operators are defined recursively by 
these rules: 


30. 


*31. 


32. 


33. 


34. 


(i) if x is a symbol, then x is a well-formed formula in 
prefix notation; 

(ii) if X and Y are well-formed formulae and * is an 
operator, then « XY is a well-formed formula. 


Which of these are well-formed formulae over the sym- 
bols {x, y, z} and the set of binary operators {x, +, o}? 
a) x++xyx 

b) oxyx xz 

ec) Xoxzxxxy 

d) xX +oxxoxxx 


Show that any well-formed formula in prefix notation 
over a set of symbols and a set of binary operators contains 
exactly one more symbol than the number of operators. 


Give a definition of well-formed formulae in postfix no- 
tation over a set of symbols and a set of binary operators. 


Give six examples of well-formed formulae with three or 
more operators in postfix notation over the set of symbols 
{x, y, z} and the set of operators {+, x, o}. 


Extend the definition of well-formed formulae in prefix 
notation to sets of symbols and operators where the op- 
erators may not be binary. 
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Etna Old Town Etna Old Town 
a b c d 
Orono Orono 
Herman’ Bangor Herman} Bangor 
Hampden Hampden 7 & 
(a) (b) 
FIGURE 1 (a) A Road System and (b) a Set FIGURE2 The 
of Roads to Plow. Simple Graph G. 


11.4 Spanning Trees 


Introduction 


Consider the system of roads in Maine represented by the simple graph shown in Figure 1 (a). 
The only way the roads can be kept open in the winter is by frequently plowing them. The 
highway department wants to plow the fewest roads so that there will always be cleared roads 
connecting any two towns. How can this be done? 

At least five roads must be plowed to ensure that there is a path between any two towns. 
Figure 1(b) shows one such set of roads. Note that the subgraph representing these roads is a 
tree, because it is connected and contains six vertices and five edges. 

This problem was solved with a connected subgraph with the minimum number of edges 
containing all vertices of the original simple graph. Such a graph must be a tree. 


DEFINITION 1 _ Let G bea simple graph. A spanning tree of G is a subgraph of G that is a tree containing 
every vertex of G. 


A simple graph with a spanning tree must be connected, because there is a path in the 
spanning tree between any two vertices. The converse is also true; that is, every connected 
simple graph has a spanning tree. We will give an example before proving this result. 


EXAMPLE1 Finda spanning tree of the simple graph G shown in Figure 2. 


Solution: The graph G is connected, but it is not a tree because it contains simple circuits. 
Remove the edge {a, e}. This eliminates one simple circuit, and the resulting subgraph is still 
connected and still contains every vertex of G. Next remove the edge {e, f} to eliminate a 
second simple circuit. Finally, remove edge {c, g} to produce a simple graph with no simple 
circuits. This subgraph is a spanning tree, because it is a tree that contains every vertex of G. 
The sequence of edge removals used to produce the spanning tree is illustrated in Figure 3. 


a b é d a b c d 
® ce @ 
e g é § 
f 
Edge removed: {a, ¢} {e, f} 


(a) (b) (c) 


FIGURE3 Producing a Spanning Tree for G by Removing Edges That Form Simple Circuits. 
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THEOREM 1 


EXAMPLE 2 


Links 


a b é d a b c d 
e g 
e f g f 
a b c d a b c d 
2 
e g 


i) 
a 
a9 
yw 


FIGURE 4 Spanning Trees of G. 


The tree shown in Figure 3 is not the only spanning tree of G. For instance, each of the trees 
shown in Figure 4 is a spanning tree of G. < 


A simple graph is connected if and only if it has a spanning tree. 


Proof: First, suppose that a simple graph G has a spanning tree T. T contains every vertex of G. 
Furthermore, there is a path in T between any two of its vertices. Because T is a subgraph of G, 
there is a path in G between any two of its vertices. Hence, G is connected. 

Now suppose that G is connected. If G is not a tree, it must contain a simple circuit. Remove 
an edge from one of these simple circuits. The resulting subgraph has one fewer edge but still 
contains all the vertices of G and is connected. This subgraph is still connected because when 
two vertices are connected by a path containing the removed edge, they are connected by a path 
not containing this edge. We can construct such a path by inserting into the original path, at 
the point where the removed edge once was, the simple circuit with this edge removed. If this 
subgraph is not a tree, it has a simple circuit; so as before, remove an edge that is in a simple 
circuit. Repeat this process until no simple circuits remain. This is possible because there are 
only a finite number of edges in the graph. The process terminates when no simple circuits 
remain. A tree is produced because the graph stays connected as edges are removed. This tree 
is a Spanning tree because it contains every vertex of G. | 


Spanning trees are important in data networking, as Example 2 shows. 


IP Multicasting Spanning trees play an important role in multicasting over Internet Protocol 
(IP) networks. To send data from a source computer to multiple receiving computers, each of 
which is a subnetwork, data could be sent separately to each computer. This type of networking, 
called unicasting, is inefficient, because many copies of the same data are transmitted over the 
network. To make the transmission of data to multiple receiving computers more efficient, IP 
multicasting is used. With IP multicasting, a computer sends a single copy of data over the 
network, and as data reaches intermediate routers, the data are forwarded to one or more other 
routers so that ultimately all receiving computers in their various subnetworks receive these data. 
(Routers are computers that are dedicated to forwarding IP datagrams between subnetworks in 
a network. In multicasting, routers use Class D addresses, each representing a session that 
receiving computers may join; see Example 17 in Section 6.1.) 

For data to reach receiving computers as quickly as possible, there should be no loops 
(which in graph theory terminology are circuits or cycles) in the path that data take through the 
network. That is, once data have reached a particular router, data should never return to this 


Links 
Demo 
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IP network Multicast spanning tree 


re © 


Source Source 


(a) (b) 


Router 


e Subnetwork 


© Subnetwork with a receiving station 


FIGURE 5 A Multicast Spanning Tree. 


router. To avoid loops, the multicast routers use network algorithms to construct a spanning tree 
in the graph that has the multicast source, the routers, and the subnetworks containing receiving 
computers as vertices, with edges representing the links between computers and/or routers. 
The root of this spanning tree is the multicast source. The subnetworks containing receiving 
computers are leaves of the tree. (Note that subnetworks not containing receiving stations are 
not included in the graph.) This is illustrated in Figure 5. < 


Depth-First Search 


The proof of Theorem | gives an algorithm for finding spanning trees by removing edges from 
simple circuits. This algorithm is inefficient, because it requires that simple circuits be identified. 
Instead of constructing spanning trees by removing edges, spanning trees can be built up by 
successively adding edges. Two algorithms based on this principle will be presented here. 

We can build a spanning tree for a connected simple graph using depth-first search. We 
will form a rooted tree, and the spanning tree will be the underlying undirected graph of this 
rooted tree. Arbitrarily choose a vertex of the graph as the root. Form a path starting at this 
vertex by successively adding vertices and edges, where each new edge is incident with the last 
vertex in the path and a vertex not already in the path. Continue adding vertices and edges to this 
path as long as possible. If the path goes through all vertices of the graph, the tree consisting of 
this path is a spanning tree. However, if the path does not go through all vertices, more vertices 
and edges must be added. Move back to the next to last vertex in the path, and, if possible, form 
a new path starting at this vertex passing through vertices that were not already visited. If this 
cannot be done, move back another vertex in the path, that is, two vertices back in the path, and 
try again. 

Repeat this procedure, beginning at the last vertex visited, moving back up the path one 
vertex at a time, forming new paths that are as long as possible until no more edges can be 
added. Because the graph has a finite number of edges and is connected, this process ends with 
the production of a spanning tree. Each vertex that ends a path at a stage of the algorithm will 
be a leaf in the rooted tree, and each vertex where a path is constructed starting at this vertex 
will be an internal vertex. 
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FIGURE 6 The Graph G. FIGURE 7 Depth-First Search of G. 


EXAMPLE 3 
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EXAMPLE 4 


The reader should note the recursive nature of this procedure. Also, note that if the vertices 
in the graph are ordered, the choices of edges at each stage of the procedure are all determined 
when we always choose the first vertex in the ordering that is available. However, we will not 
always explicitly order the vertices of a graph. 

Depth-first search is also called backtracking, because the algorithm returns to vertices 
previously visited to add paths. Example 3 illustrates backtracking. 


Use depth-first search to find a spanning tree for the graph G shown in Figure 6. 


Solution: The steps used by depth-first search to produce a spanning tree of G are shown in 
Figure 7. We arbitrarily start with the vertex f. A path is built by successively adding edges 
incident with vertices not already in the path, as long as this is possible. This produces a path 
ft, g,h, k, j (note that other paths could have been built). Next, backtrack to k. There is no 
path beginning at k containing vertices not already visited. So we backtrack to h. Form the 
path h, i. Then backtrack to h, and then to f. From f build the path f, d, e, c, a. Then backtrack 
to c and form the path c, b. This produces the spanning tree. < 


The edges selected by depth-first search of a graph are called tree edges. All other edges 
of the graph must connect a vertex to an ancestor or descendant of this vertex in the tree. These 
edges are called back edges. (Exercise 43 asks for a proof of this fact.) 


In Figure 8 we highlight the tree edges found by depth-first search starting at vertex f by showing 
them with heavy colored lines. The back edges (e, f) and (f, 2) are shown with thinner black 
lines. < 


We have explained how to find a spanning tree of a graph using depth-first search. However, 
our discussion so far has not brought out the recursive nature of depth-first search. To help 
make the recursive nature of the algorithm clear, we need a little terminology. We say that we 


b g 


FIGURE 8 The Tree Edges and Back Edges 
of the Depth-First Search in Example 4. 
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explore from a vertex v when we carry out the steps of depth-first search beginning when v is 
added to the tree and ending when we have backtracked back to v for the last time. The key 
observation needed to understand the recursive nature of the algorithm is that when we add an 
edge connecting a vertex v to a vertex w, we finish exploring from w before we return to v to 
complete exploring from v. 

In Algorithm 1 we construct the spanning tree of a graph G with vertices v;,..., v, by first 
selecting the vertex v, to be the root. We initially set T to be the tree with just this one vertex. 
At each step we add a new vertex to the tree T together with an edge from a vertex already 
in T to this new vertex and we explore from this new vertex. Note that at the completion of 
the algorithm, T contains no simple circuits because no edge is ever added that connects two 
vertices in the tree. Moreover, T remains connected as it is built. (These last two observations 
can be easily proved via mathematical induction.) Because G is connected, every vertex in G 
is visited by the algorithm and is added to the tree (as the reader should verify). It follows 
that T is a spanning tree of G. 


ALGORITHM 1 Depth-First Search. 


procedure DFS(G: connected graph with vertices vj, v2, ..., Vn) 
T := tree consisting only of the vertex v1 
visit(v1) 


procedure visit(v: vertex of G) 

for each vertex w adjacent to v and not yet in T 
add vertex w and edge {v, w} to T 
visit(w) 


We now analyze the computational complexity of the depth-first search algorithm. The key 
observation is that for each vertex v, the procedure visit(v) is called when the vertex v is first 
encountered in the search and it is not called again. Assuming that the adjacency lists for G are 
available (see Section 10.3), no computations are required to find the vertices adjacent to v. As 
we follow the steps of the algorithm, we examine each edge at most twice to determine whether 
to add this edge and one of its endpoints to the tree. Consequently, the procedure DFS constructs 
a spanning tree using O(e), or O(n”), steps where e and n are the number of edges and vertices 
in G, respectively. [Note that a step involves examining a vertex to see whether it is already in 
the spanning tree as it is being built and adding this vertex and the corresponding edge if the 
vertex is not already in the tree. We have also made use of the inequality e < n(n — 1)/2, which 
holds for any simple graph. ] 

Depth-first search can be used as the basis for algorithms that solve many different problems. 
For example, it can be used to find paths and circuits in a graph, it can be used to determine 
the connected components of a graph, and it can be used to find the cut vertices of a connected 
graph. As we will see, depth-first search is the basis of backtracking techniques used to search 
for solutions of computationally difficult problems. (See [GrYe05], [Ma89], and [CoLeRiSt09] 
for a discussion of algorithms based on depth-first search.) 


Breadth-First Search 


We can also produce a spanning tree of a simple graph by the use of breadth-first search. 
Again, a rooted tree will be constructed, and the underlying undirected graph of this rooted tree 
forms the spanning tree. Arbitrarily choose a root from the vertices of the graph. Then add all 
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FIGURE 9 A Graph G. 


edges incident to this vertex. The new vertices added at this stage become the vertices at level | 
in the spanning tree. Arbitrarily order them. Next, for each vertex at level 1, visited in order, 
add each edge incident to this vertex to the tree as long as it does not produce a simple circuit. 
Arbitrarily order the children of each vertex at level 1. This produces the vertices at level 2 
in the tree. Follow the same procedure until all the vertices in the tree have been added. The 
procedure ends because there are only a finite number of edges in the graph. A spanning tree is 
produced because we have produced a tree containing every vertex of the graph. An example of 
breadth-first search is given in Example 5. 


Use breadth-first search to find a spanning tree for the graph shown in Figure 9. 


Solution: The steps of the breadth-first search procedure are shown in Figure 10. We choose 
the vertex e to be the root. Then we add edges incident with all vertices adjacent to e, so edges 
from e to b, d, f, andi are added. These four vertices are at level | in the tree. Next, add the 
edges from these vertices at level 1 to adjacent vertices not already in the tree. Hence, the edges 
from b to a and c are added, as are edges from d toh, from f to j and g, and from i to k. The 
new vertices a, c,h, j, g, and k are at level 2. Next, add edges from these vertices to adjacent 
vertices not already in the graph. This adds edges from g to/ and from k to m. 


We describe breadth-first search in pseudocode as Algorithm 2. In this algorithm, we assume 
the vertices of the connected graph G are ordered as vj, v2, ..., Vy. In the algorithm we use the 
term “process” to describe the procedure of adding new vertices, and corresponding edges, to 
the tree adjacent to the current vertex being processed as long as a simple circuit is not produced. 


b d f i 
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FIGURE 10 Breadth-First Search of G. 
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ALGORITHM 2 Breadth-First Search. 


procedure BFS (G: connected graph with vertices vj, v2, ..., Vn) 
T := tree consisting only of vertex v, 
L := empty list 
put v1 in the list ZL of unprocessed vertices 
while L is not empty 
remove the first vertex, v, from L 
for each neighbor w of v 
if w is not in Z and not in T then 
add w to the end of the list L 
add w and edge {v, w} to T 


We now analyze the computational complexity of breadth-first search. For each vertex v in 
the graph we examine all vertices adjacent to v and we add each vertex not yet visited to the 
tree T. Assuming we have the adjacency lists for the graph available, no computation is required 
to determine which vertices are adjacent to a given vertex. As in the analysis of the depth-first 
search algorithm, we see that we examine each edge at most twice to determine whether we 
should add this edge and its endpoint not already in the tree. It follows that the breadth-first 
search algorithm uses O(e) or O(n”) steps. 

Breadth-first search is one of the most useful algorithms in graph theory. In particular, it can 
serve as the basis for algorithms that solve a wide variety of problems. For example, algorithms 
that find the connected components of a graph, that determine whether a graph is bipartite, and 
that find the path with the fewest edges between two vertices in a graph can all be built using 
breadth-first search. 


Backtracking Applications 


There are problems that can be solved only by performing an exhaustive search of all possible 
solutions. One way to search systematically for a solution is to use a decision tree, where each 
internal vertex represents a decision and each leaf a possible solution. To find a solution via 
backtracking, first make a sequence of decisions in an attempt to reach a solution as long as this 
is possible. The sequence of decisions can be represented by a path in the decision tree. Once 
it is known that no solution can result from any further sequence of decisions, backtrack to the 
parent of the current vertex and work toward a solution with another series of decisions, if this is 
possible. The procedure continues until a solution is found, or it is established that no solution 
exists. Examples 6 to 8 illustrate the usefulness of backtracking. 


Graph Colorings How can backtracking be used to decide whether a graph can be colored 
using colors? 


Solution: We can solve this problem using backtracking in the following way. First pick some 
vertex a and assign it color 1. Then pick a second vertex b, and if b is not adjacent to a, assign 
it color 1. Otherwise, assign color 2 to b. Then go on to a third vertex c. Use color 1, if possible, 
for c. Otherwise use color 2, if this is possible. Only if neither color | nor color 2 can be used 
should color 3 be used. Continue this process as long as it is possible to assign one of the n 
colors to each additional vertex, always using the first allowable color in the list. If a vertex is 
reached that cannot be colored by any of the n colors, backtrack to the last assignment made and 
change the coloring of the last vertex colored, if possible, using the next allowable color in the 
list. If it is not possible to change this coloring, backtrack farther to previous assignments, one 
step back at a time, until it is possible to change a coloring of a vertex. Then continue assigning 
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FIGURE 11 Coloring a Graph Using Backtracking. 


colors of additional vertices as long as possible. If a coloring using n colors exists, backtracking 
will produce it. (Unfortunately this procedure can be extremely inefficient.) 

In particular, consider the problem of coloring the graph shown in Figure 11 with three 
colors. The tree shown in Figure 11 illustrates how backtracking can be used to construct a 
3-coloring. In this procedure, red is used first, then blue, and finally green. This simple example 
can obviously be done without backtracking, but it is a good illustration of the technique. 

In this tree, the initial path from the root, which represents the assignment of red to a, leads 
to a coloring with a red, b blue, c red, and d green. It is impossible to color e using any of 
the three colors when a, b, c, and d are colored in this way. So, backtrack to the parent of the 
vertex representing this coloring. Because no other color can be used for d, backtrack one more 
level. Then change the color of c to green. We obtain a coloring of the graph by then assigning 
red to d and green to e. < 


The m-Queens Problem The n-queens problem asks how n queens can be placed on ann x n 
chessboard so that no two queens can attack one another. How can backtracking be used to solve 
the n-queens problem? 


Solution: To solve this problem we must find n positions on an n x n chessboard so that no 
two of these positions are in the same row, same column, or in the same diagonal [a diagonal 
consists of all positions (7, 7) with i + j = m for some m, ori — j = m for some m]. We will 
use backtracking to solve the n-queens problem. We start with an empty chessboard. At stage 
k + 1 we attempt putting an additional queen on the board in the (k + 1)st column, where there 
are already queens in the first k columns. We examine squares in the (k + 1)st column starting 
with the square in the first row, looking for a position to place this queen so that it is not in the 
same row or on the same diagonal as a queen already on the board. (We already know it is not 
in the same column.) If it is impossible to find a position to place the queen in the (k + 1)st 
column, backtrack to the placement of the queen in the kth column, and place this queen in the 
next allowable row in this column, if such a row exists. If no such row exists, backtrack further. 

In particular, Figure 12 displays a backtracking solution to the four-queens problem. In this 
solution, we place a queen in the first row and column. Then we put a queen in the third row of 
the second column. However, this makes it impossible to place a queen in the third column. So 
we backtrack and put a queen in the fourth row of the second column. When we do this, we can 
place a queen in the second row of the third column. But there is no way to add a queen to the 
fourth column. This shows that no solution results when a queen is placed in the first row and 
column. We backtrack to the empty chessboard, and place a queen in the second row of the first 
column. This leads to a solution as shown in Figure 12. < 
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FIGURE 12 A Backtracking Solution of the Four-Queens Problem. 


EXAMPLE 8 Sums of Subsets Consider this problem. Given a set of positive integers x;, x2,..., Xn, find 
a subset of this set of integers that has M as its sum. How can backtracking be used to solve this 
problem? 


Solution: We start with a sum with no terms. We build up the sum by successively adding terms. 
An integer in the sequence is included if the sum remains less than M when this integer is added 
to the sum. If a sum is reached such that the addition of any term is greater than M, backtrack 
by dropping the last term of the sum. 

Figure 13 displays a backtracking solution to the problem of finding a subset of 
{31, 27, 15, 11, 7, 5} with the sum equal to 39. < 


Sum = 0 


{31, 7} {31,5} {27, 11} {27, 7} 
Sum = 38 Sum = 36 Sum = 38 Sum = 34 


{27, 7, 5} 
Sum = 39 


FIGURE 13 Find a Sum Equal to 39 Using Backtracking. 
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FIGURE 14  Depth-First Search of a Directed Graph. 


Depth-First Search in Directed Graphs 


We can easily modify both depth-first search and breadth-first search so that they can run given 
a directed graph as input. However, the output will not necessarily be a spanning tree, but rather 
a spanning forest. In both algorithms we can add an edge only when it is directed away from 
the vertex that is being visited and to a vertex not yet added. If at a stage of either algorithm we 
find that no edge exists starting at a vertex already added to one not yet added, the next vertex 
added by the algorithm becomes the root of a new tree in the spanning forest. This is illustrated 
in Example 9. 


What is the output of depth-first search given the graph G shown in Figure 14(a) as input? 


Solution: We begin the depth-first search at vertex a and add vertices b, c, and g and the 
corresponding edges where we are blocked. We backtrack to c but we are still blocked, and then 
backtrack to b, where we add vertices f and e and the corresponding edges. Backtracking takes 
us all the way back to a. We then start a new tree at d and add vertices h,/, k, and j and the 
corresponding edges. We backtrack to k, then /, then h, and back to d. Finally, we start a new 
tree at i, completing the depth-first search. The output is shown in Figure 14(b). < 


Depth-first search in directed graphs is the basis of many algorithms (see [GrYe05], [Ma89], 
and [CoLeRiSt09]). It can be used to determine whether a directed graph has a circuit, it can 
be used to carry out a topological sort of a graph, and it can also be used to find the strongly 
connected components of a directed graph. 

We conclude this section with an application of depth-first search and breadth-first search 
to search engines on the Web. 


Web Spiders To index websites, search engines such as Google and Yahoo systematically 
explore the Web starting at known sites. These search engines use programs called Web spiders 
(or crawlers or bots) to visit websites and analyze their contents. Web spiders use both depth-first 
searching and breadth-first searching to create indices. As described in Example 5 in Section 10.1, 
Web pages and links between them can be modeled by a directed graph called the Web graph. 
Web pages are represented by vertices and links are represented by directed edges. Using depth- 
first search, an initial Web page is selected, a link is followed to a second Web page (if there is 
such a link), a link on the second Web page is followed to a third Web page, if there is such a 
link, and so on, until a page with no new links is found. Backtracking is then used to examine 
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links at the previous level to look for new links, and so on. (Because of practical limitations, Web 
spiders have limits to the depth they search in depth-first search.) Using breadth-first search, an 
initial Web page is selected and a link on this page is followed to a second Web page, then a 
second link on the initial page is followed (if it exists), and so on, until all links of the initial 
page have been followed. Then links on the pages one level down are followed, page by page, 


and so on. 


Exercises 


1. How many edges must be removed from a connected 
graph with n vertices and m edges to produce a spanning 
tree? 

In Exercises 2-6 find a spanning tree for the graph shown by 
removing edges in simple circuits. 


2. 4 b c 
d e 
3. 
d 
&§ 
4, 


7. Find a spanning tree for each of these graphs. 
a) Ks b) K44 c) Ki6 
d) Q3 e) Cs f) Ws 
In Exercises 8-10 draw all the spanning trees of the given 
simple graphs. 


8. a 
b c 
9, a b f g 
c d e h 
10. 


*11. How many different spanning trees does each of these 
simple graphs have? 
a) K3 b) K4 c) K22 d) Cs 
*12. How many nonisomorphic spanning trees does each of 
these simple graphs have? 
a) K3 b) K4 c) Ks 
In Exercises 13-15 use depth-first search to produce a span- 
ning tree for the given simple graph. Choose a as the root of 
this spanning tree and assume that the vertices are ordered 
alphabetically. 


13. a e h i 


796 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24, 


* 25, 
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q 


ia} 


b iy 


n m 
Use breadth-first search to produce a spanning tree for 
each of the simple graphs in Exercises 13-15. Choose a 
as the root of each spanning tree. 


Use depth-first search to find a spanning tree of each of 
these graphs. 


a) We (see Example 7 of Section 10.2), starting at the 
vertex of degree 6 

b) Ks 

c) K3,4, starting at a vertex of degree 3 

d) Q3 

Use breadth-first search to find a spanning tree of each of 

the graphs in Exercise 17. 

Describe the trees produced by breadth-first search and 

depth-first search of the wheel graph W,,, starting at the 

vertex of degree n, where n is an integer withn > 3. (See 

Example 7 of Section 10.2.) Justify your answers. 


Describe the trees produced by breadth-first search and 
depth-first search of the complete graph K,,, where n is a 
positive integer. Justify your answers. 


Describe the trees produced by breadth-first search and 
depth-first search of the complete bipartite graph Kim,n, 
starting at a vertex of degree m, where m and n are positive 
integers. Justify your answers. 


Describe the tree produced by breadth-first search and 
depth-first search for the n-cube graph Q,, where n is a 
positive integer. 

Suppose that an airline must reduce its flight schedule to 
save money. If its original routes are as illustrated here, 
which flights can be discontinued to retain service be- 
tween all pairs of cities (where it may be necessary to 
combine flights to fly from one city to another)? 


Bangor 
Detroit 
Seattle : Boston 
. 
San ee SS ae York 
Francisco . 
Washington 
Los 
Angeles Atlanta 


Explain how breadth-first search or depth-first search can 
be used to order the vertices of a connected graph. 
Show that the length of the shortest path between ver- 
tices v and u in aconnected simple graph equals the level 
number of u in the breadth-first spanning tree of G with 
root v. 


26. 


27. 


28. 


29. 


30. 


Use backtracking to try to find a coloring of each of the 
graphs in Exercises 7—9 of Section 10.8 using three col- 
ors. 


— 


Use backtracking to solve the n-queens problem for these 
values of n. 

a)n=3 b) n=5 c) n=6 

Use backtracking to find a subset, if it exists, of the set 
{27, 24, 19, 14, 11, 8} with sum 

a) 20. b) 41. c) 60. 

Explain how backtracking can be used to find a Hamilton 


path or circuit in a graph. 


a) Explain how backtracking can be used to find the way 
out of a maze, given a starting position and the exit 
position. Consider the maze divided into positions, 
where at each position the set of available moves in- 
cludes one to four possibilities (up, down, right, left). 

b) Find a path from the starting position marked by X to 
the exit in this maze. 


x 


Exit 


A spanning forest of a graph G is a forest that contains every 
vertex of G such that two vertices are in the same tree of the 
forest when there is a path in G between these two vertices. 


31. 
32. 
33. 


34. 


35. 


36. 


37. 


38. 


39. 


40. 


Show that every finite simple graph has a spanning forest. 
How many trees are in the spanning forest of a graph? 


How many edges must be removed to produce the span- 
ning forest of a graph with n vertices, m edges, and c 
connected components? 


Let G be a connected graph. Show that if T is a spanning 
tree of G constructed using breadth-first search, then an 
edge of G not in T must connect vertices at the same level 
or at levels that differ by 1 in this spanning tree. 


Explain how to use breadth-first search to find the length 
of a shortest path between two vertices in an undirected 
graph. 

Devise an algorithm based on breadth-first search that de- 
termines whether a graph has a simple circuit, and if so, 
finds one. 


Devise an algorithm based on breadth-first search for find- 
ing the connected components of a graph. 


Explain how breadth-first search and how depth-first 
search can be used to determine whether a graph is bipar- 
tite. 


Which connected simple graphs have exactly one span- 
ning tree? 
Devise an algorithm for constructing the spanning for- 


est of a graph based on deleting edges that form simple 
circuits. 


41. 
42. 


43. 


44. 


45. 


46. 
47. 


48. 


49. 


*50. 


51. 


*52, 


Minimum Spanning Trees 


Devise an algorithm for constructing the spanning forest 
of a graph based on depth-first searching. 

Devise an algorithm for constructing the spanning forest 
of a graph based on breadth-first searching. 

Let G be a connected graph. Show that if T is a span- 
ning tree of G constructed using depth-first search, then 
an edge of G not in T must be a back edge, that is, it 
must connect a vertex to one of its ancestors or one of its 
descendants in T. 


When must an edge of a connected simple graph be in 
every spanning tree for this graph? 

For which graphs do depth-first search and breadth-first 
search produce identical spanning trees no matter which 
vertex is selected as the root of the tree? Justify your an- 
swet. 

Use Exercise 43 to prove that if G is a connected, simple 
graph with n vertices and G does not contain a simple 
path of length k then it contains at most (k — 1)n edges. 
Use mathematical induction to prove that breadth-first 
search visits vertices in order of their level in the result- 
ing spanning tree. 

Use pseudocode to describe a variation of depth-first 
search that assigns the integer n to the nth vertex vis- 
ited in the search. Show that this numbering corresponds 
to the numbering of the vertices created by a preorder 
traversal of the spanning tree. 

Use pseudocode to describe a variation of breadth-first 
search that assigns the integer m to the mth vertex visited 
in the search. 

Suppose that G is a directed graph and T is a spanning 
tree constructed using breadth-first search. Show that ev- 
ery edge of G has endpoints that are at the same level or 
one level higher or lower. 

Show that if G is a directed graph and T is a spanning 
tree constructed using depth-first search, then every edge 
not in the spanning tree is a forward edge connecting 
an ancestor to a descendant, a back edge connecting a 
descendant to an ancestor, or a cross edge connecting a 
vertex to a vertex in a previously visited subtree. 
Describe a variation of depth-first search that assigns the 
smallest available positive integer to a vertex when the 
algorithm is totally finished with this vertex. Show that in 
this numbering, each vertex has a larger number than its 
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children and that the children have increasing numbers 
from left to right. 
Let 7; and 7> be spanning trees of a graph. The distance be- 
tween 7) and 7> is the number of edges in 7; and 7» that are 
not common to 7; and 7). 


53. Find the distance between each pair of spanning trees 
shown in Figures 3(c) and 4 of the graph G shown in 
Figure 2. 


*54, Suppose that 7|, T2, and 73 are spanning trees of the 
simple graph G. Show that the distance between 7; 
and T3 does not exceed the sum of the distance between T, 
and T> and the distance between 7> and 73. 


** 55, Suppose that 7; and 7> are spanning trees of a simple 


graph G. Moreover, suppose that e; is an edge in 7] that 
is not in 77. Show that there is an edge e2 in T> that is not 
in T; such that T; remains a spanning tree if e; is removed 
from it and e2 is added to it, and 7 remains a spanning 
tree if e2 is removed from it and e; is added to it. 


*56. Show that it is possible to find a sequence of spanning 
trees leading from any spanning tree to any other by suc- 
cessively removing one edge and adding another. 


A rooted spanning tree of a directed graph is a rooted tree 
containing edges of the graph such that every vertex of the 
graph is an endpoint of one of the edges in the tree. 


57. For each of the directed graphs in Exercises 18—23 of Sec- 
tion 10.5 either find a rooted spanning tree of the graph 
or determine that no such tree exists. 


*58. Show that a connected directed graph in which each ver- 
tex has the same in-degree and out-degree has a rooted 
spanning tree. [Hint: Use an Euler circuit.] 


*59. Give an algorithm to build a rooted spanning tree for con- 
nected directed graphs in which each vertex has the same 
in-degree and out-degree. 


*60. Show that if G is a directed graph and T is a spanning 
tree constructed using depth-first search, then G contains 
a circuit if and only if G contains a back edge (see Exer- 
cise 51) relative to the spanning tree T. 


*61. Use Exercise 60 to construct an algorithm for determining 
whether a directed graph contains a circuit. 


Introduction 


Links 


A company plans to build a communications network connecting its five computer centers. Any 
pair of these centers can be linked with a leased telephone line. Which links should be made to 


ensure that there is a path between any two computer centers so that the total cost of the network 
is minimized? We can model this problem using the weighted graph shown in Figure 1, where 
vertices represent computer centers, edges represent possible leased lines, and the weights on 
edges are the monthly lease rates of the lines represented by the edges. We can solve this problem 
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$1200 
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FIGURE 1 A Weighted Graph Showing Monthly Lease 
Costs for Lines in a Computer Network. 


by finding a spanning tree so that the sum of the weights of the edges of the tree is minimized. 
Such a spanning tree is called a minimum spanning tree. 


Algorithms for Minimum Spanning Trees 


A wide variety of problems are solved by finding a spanning tree in a weighted graph such that 
the sum of the weights of the edges in the tree is a minimum. 


A minimum spanning tree in a connected weighted graph is a spanning tree that has the 
smallest possible sum of weights of its edges. 


We will present two algorithms for constructing minimum spanning trees. Both proceed by 
successively adding edges of smallest weight from those edges with a specified property that 
have not already been used. Both are greedy algorithms. Recall from Section 3.1 that a greedy 
algorithm is a procedure that makes an optimal choice at each of its steps. Optimizing at each step 
does not guarantee that the optimal overall solution is produced. However, the two algorithms 
presented in this section for constructing minimum spanning trees are greedy algorithms that 
do produce optimal solutions. 

The first algorithm that we will discuss was originally discovered by the Czech mathe- 
matician Vojtéch Jarnik in 1930, who described it in a paper in an obscure Czech journal. The 
algorithm became well known when it was rediscovered in 1957 by Robert Prim. Because of 
this, it is known as Prim’s algorithm (and sometimes as the Prim-Jarnik algorithm). Begin 
by choosing any edge with smallest weight, putting it into the spanning tree. Successively add to 
the tree edges of minimum weight that are incident to a vertex already in the tree, never forming 
a simple circuit with those edges already in the tree. Stop when n — | edges have been added. 

Later in this section, we will prove that this algorithm produces a minimum spanning tree for 
any connected weighted graph. Algorithm | gives a pseudocode description of Prim’s algorithm. 


ROBERT CLAY PRIM (BORN 1921) Robert Prim, born in Sweetwater, Texas, received his B.S. in electrical 
engineering in 1941 and his Ph.D. in mathematics from Princeton University in 1949. He was an engineer at 
the General Electric Company from 1941 until 1944, an engineer and mathematician at the United States Naval 
Ordnance Lab from 1944 until 1949, and a research associate at Princeton University from 1948 until 1949. 
Among the other positions he has held are director of mathematics and mechanics research at Bell Telephone 
Laboratories from 1958 until 1961 and vice president of research at Sandia Corporation. He is currently retired. 


San Francisco 
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New York 


Choice Edge Cost 
1 {Chicago, Atlanta} $ 700 
2 {Atlanta, New York} $ 800 
3 {Chicago, San Francisco} $1200 
4 {San Francisco, Denver} $ 900 
Total: $3600 i j k l 
FIGURE 2 A Minimum Spanning Tree for the Weighted FIGURE 3 A Weighted Graph. 
Graph in Figure 1. 
ALGORITHM 1 Prim’s Algorithm. 
procedure Prim(G: weighted connected undirected graph with n vertices) 
T := a minimum-weight edge 
fori := 1 ton—2 
e := an edge of minimum weight incident to a vertex in T and not forming a 
simple circuit in T if added to T 
T :=T with e added 
return 7 {T is a minimum spanning tree of G} 
Note that the choice of an edge to add at a stage of the algorithm is not determined when there 
is more than one edge with the same weight that satisfies the appropriate criteria. We need to 
order the edges to make the choices deterministic. We will not worry about this in the remainder 
of the section. Also note that there may be more than one minimum spanning tree for a given 
connected weighted simple graph. (See Exercise 9.) Examples | and 2 illustrate how Prim’s 
algorithm is used. 
EXAMPLE1 Use Prim’s algorithm to design a minimum-cost communications network connecting all the 
computers represented by the graph in Figure 1. 
Solution: We solve this problem by finding a minimum spanning tree in the graph in Figure 1. 
Prim’s algorithm is carried out by choosing an initial edge of minimum weight and successively 
adding edges of minimum weight that are incident to a vertex in the tree and that do not form 
simple circuits. The edges in color in Figure 2 show a minimum spanning tree produced by 
Prim’s algorithm, with the choice made at each step displayed. < 
EXAMPLE 2 Use Prim’s algorithm to find a minimum spanning tree in the graph shown in Figure 3. 


Links 


Solution: A minimum spanning tree constructed using Prim’s algorithm is shown in Figure 4. 
The successive edges chosen are displayed. 


The second algorithm we will discuss was discovered by Joseph Kruskal in 1956, although 
the basic ideas it uses were described much earlier. To carry out Kruskal’s algorithm, choose 
an edge in the graph with minimum weight. 
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Choice Edge Weight 

a 2 b 3 c 1 d 1 b, f 1 
e e e ry 2 fas Bi 4 
3 {fi} 2 

3 1 2 5 4 a, e 3 
5 i, j 3 

4 f 3. «C8 3 6 fhe 3 

ee ® e eh 

7 {c, g} 2 

8 {c, d} 1 

4 2 4 3 9 gh 3 
10 hl 3 
a ee a ee eee | ul a 1 
i j k l Total: 24 


(a) (b) 


FIGURE 4 A Minimum Spanning Tree Produced Using Prim’s Algorithm. 


Successively add edges with minimum weight that do not form a simple circuit with those 
edges already chosen. Stop after n — 1 edges have been selected. 

The proof that Kruskal’s algorithm produces a minimum spanning tree for every connected 
weighted graph is left as an exercise. Pseudocode for Kruskal’s algorithm is given in Algorithm 2. 


ALGORITHM 2 Kruskal’s Algorithm. 


procedure Kruskal(G: weighted connected undirected graph with n vertices) 
T := empty graph 
fori := 1ton—1 
e := any edge in G with smallest weight that does not form a simple circuit 
when added to T 
T :=T with e added 
return 7 {T is a minimum spanning tree of G} 


JOSEPH BERNARD KRUSKAL (1928-2010) Joseph Kruskal was born in New York City, where his father 
was a fur dealer and his mother promoted the art of origami on early television. Kruskal attended the University 
of Chicago and received his Ph.D. from Princeton University in 1954. He was an instructor in mathematics 
at Princeton and at the University of Wisconsin, and later he was an assistant professor at the University of 
Michigan. In 1959 he became a member of the technical staff at Bell Laboratories, where he worked until his 
retirement in the late 1990s. Kruskal discovered his algorithm for producing minimum spanning trees when 
he was a second-year graduate student. He was not sure his 24-page paper on this subject was worthy of 
publication, but was convinced by others to submit it. His research interests included statistical linguistics 
and psychometrics. Besides his work on minimum spanning trees, Kruskal is also known for contributions to 
multidimensional scaling. It is noteworthy that Joseph Kruskal’s two brothers, Martin and William, also were 
well known mathematicians. 


HISTORICAL NOTE Joseph Kruskal and Robert Prim developed their algorithms for constructing minimum 
links MAX spanning trees in the mid-1950s. However, they were not the first people to discover such algorithms. For 
example, the work of the anthropologist Jan Czekanowski, in 1909, contains many of the ideas required to find 
minimum spanning trees. In 1926, Otakar Boruvka described methods for constructing minimum spanning trees 
in work relating to the construction of electric power networks, and as mentioned in the text what is now called 

Prim’s algorithm was discovered by Vojtéch Jarnik in 1930. 
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Choice Edge Weight 

a 2 b 3 c 1 d 1 {c, d} 1 
e © ° ° 2 {k, 1} 1 
3 {b, f} 1 

3 1 2 5 4 {c, g} 2 
5 {a, b} 2 

ee bs im 2 = 3 oh . {hj} - 
A {b, c} 3 

8 {j,k} 3 

4 2 4 3 9 {g, h} 3 
10 (ci, 3 
e : e 3 e | e 11 {a, e} 3) 
i j k ! Total: 24 


(a) (b) 


FIGURE 5 A Minimum Spanning Tree Produced by Kruskal’s Algorithm. 


The reader should note the difference between Prim’s and Kruskal’s algorithms. In Prim’s 
algorithm edges of minimum weight that are incident to a vertex already in the tree, and not 
forming a circuit, are chosen; whereas in Kruskal’s algorithm edges of minimum weight that 
are not necessarily incident to a vertex already in the tree, and that do not form a circuit, are 
chosen. Note that as in Prim’s algorithm, if the edges are not ordered, there may be more than 
one choice for the edge to add at a stage of this procedure. Consequently, the edges need to be 
ordered for the procedure to be deterministic. Example 3 illustrates how Kruskal’s algorithm is 
used. 


Use Kruskal’s algorithm to find a minimum spanning tree in the weighted graph shown in 
Figure 3. 


Solution: Aminimum spanning tree and the choices of edges at each stage of Kruskal’s algorithm 
are shown in Figure 5. < 


We will now prove that Prim’s algorithm produces a minimum spanning tree of a connected 
weighted graph. 


Proof: Let G be a connected weighted graph. Suppose that the successive edges chosen by 


Prim’s algorithm are e1, e2,..., €n—1. Let S be the tree with e), e2,..., @n—1 as its edges, and 
let S, be the tree with e), e2,...,e4 as its edges. Let T be a minimum spanning tree of G 
containing the edges e1, e2,..., ex, where k is the maximum integer with the property that a 


minimum spanning tree exists containing the first k edges chosen by Prim’s algorithm. The 
theorem follows if we can show that S = T. 

Suppose that S$ #4 7, so that k <n—1. Consequently, T contains e1, e2,..., ex, but 
not ex41. Consider the graph made up of 7 together with ex+1. Because this graph is con- 
nected and has n edges, too many edges to be a tree, it must contain a simple circuit. This simple 
circuit must contain ex, , because there was no simple circuit in 7. Furthermore, there must be 
an edge in the simple circuit that does not belong to Sx+41 because S;+1 is a tree. By starting 
at an endpoint of ex; that is also an endpoint of one of the edges e),..., ex, and following 
the circuit until it reaches an edge not in S;41, we can find an edge e not in S;,4 that has an 
endpoint that is also an endpoint of one of the edges e1, e2,..., ex. 

By deleting e from T and adding e+), we obtain a tree T’ with n — 1 edges (it is a tree be- 
cause it has no simple circuits). Note that the tree T’ contains ¢1, @2,..., ex, ex41. Furthermore, 
because ex) was chosen by Prim’s algorithm at the (k + 1)st step, and e was also available at 
that step, the weight of e,+1 is less than or equal to the weight of e. From this observation, it 
follows that T’ is also a minimum spanning tree, because the sum of the weights of its edges 
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does not exceed the sum of the weights of the edges of 7. This contradicts the choice of k as 


the maximum integer such that a minimum spanning tree exists containing e, . 


.., eg. Hence, 


k =n—1,and S =T. It follows that Prim’s algorithm produces a minimum spanning tree. <1 


It can be shown (see [CoLeRiSt09]) that to find a minimum spanning tree of a graph with m 
edges and n vertices, Kruskal’s algorithm can be carried out using O(m log m) operations and 
Prim’s algorithm can be carried out using O(m log n) operations. Consequently, it is preferable 
to use Kruskal’s algorithm for graphs that are sparse, that is, where m is very small compared to 
C(n, 2) = n(n — 1)/2, the total number of possible edges in an undirected graph with n vertices. 
Otherwise, there is little difference in the complexity of these two algorithms. 


The roads represented by this graph are all unpaved. The 
lengths of the roads between pairs of towns are repre- 
sented by edge weights. Which roads should be paved 
so that there is a path of paved roads between each 
pair of towns so that a minimum road length is paved? 
(Note: These towns are in Nevada.) 


Manhattan 


Tonopah Warm Springs 


Beatty 


In Exercises 2-4 use Prim’s algorithm to find a minimum 
spanning tree for the given weighted graph. 


10. 


ercises 


. Use Kruskal’s algorithm to design the communications 


network described at the beginning of the section. 


. Use Kruskal’s algorithm to find a minimum spanning tree 


for the weighted graph in Exercise 2. 


. Use Kruskal’s algorithm to find a minimum spanning tree 


for the weighted graph in Exercise 3. 


. Use Kruskal’s algorithm to find a minimum spanning tree 


for the weighted graph in Exercise 4. 


. Find a connected weighted simple graph with the fewest 


edges possible that has more than one minimum spanning 
tree. 


A minimum spanning forest in a weighted graph is a 
spanning forest with minimal weight. Explain how Prim’s 
and Kruskal’s algorithms can be adapted to construct min- 
imum spanning forests. 


A maximum spanning tree of a connected weighted undi- 
rected graph is a spanning tree with the largest possible weight. 


11. 


12. 


13. 


14. 


Devise an algorithm similar to Prim’s algorithm for 
constructing a maximum spanning tree of a connected 
weighted graph. 

Devise an algorithm similar to Kruskal’s algorithm for 
constructing a maximum spanning tree of a connected 
weighted graph. 

Find a maximum spanning tree for the weighted graph in 
Exercise 2. 


Find a maximum spanning tree for the weighted graph in 
Exercise 3. 


15. Find a maximum spanning tree for the weighted graph in 
Exercise 4. 

16. Find the second least expensive communications network 
connecting the five computer centers in the problem posed 
at the beginning of the section. 

*17. Devise an algorithm for finding the second shortest span- 
ning tree in a connected weighted graph. 

*18. Show that an edge with smallest weight in a connected 
weighted graph must be part of any minimum spanning 
tree. 

19. Show that there is a unique minimum spanning tree in a 
connected weighted graph if the weights of the edges are 
all different. 

20. Suppose that the computer network connecting the cities 
in Figure | must contain a direct link between New York 
and Denver. What other links should be included so that 
there is a link between every two computer centers and 
the cost is minimized? 

21. Find a spanning tree with minimal total weight contain- 
ing the edges {e, i} and {g, k} in the weighted graph in 
Figure 3. 

22. Describe an algorithm for finding a spanning tree with 
minimal weight containing a specified set of edges in a 
connected weighted undirected simple graph. 

23. Express the algorithm devised in Exercise 22 in pseu- 
docode. 


Sollin’s algorithm produces a minimum spanning tree from 
a connected weighted simple graph G = (V, E) by succes- 
sively adding groups of edges. Suppose that the vertices in V 
are ordered. This produces an ordering of the edges where 
{uo, Vo} precedes {uv 1, vy} if uo precedes uw, or if ug = uy 
and vo precedes vj. The algorithm begins by simultaneously 
choosing the edge of least weight incident to each vertex. The 
first edge in the ordering is taken in the case of ties. This pro- 
duces a graph with no simple circuits, that is, a forest of trees 
(Exercise 24 asks for a proof of this fact). Next, simultaneously 
choose for each tree in the forest the shortest edge between a 
vertex in this tree and a vertex in a different tree. Again the first 
edge in the ordering is chosen in the case of ties. (This pro- 
duces a graph with no simple circuits containing fewer trees 
than were present before this step; see Exercise 24.) Continue 
the process of simultaneously adding edges connecting trees 
until n — 1 edges have been chosen. At this stage a minimum 
spanning tree has been constructed. 


Key Terms and Results 


TERMS 
tree: a connected undirected graph with no simple circuits 
forest: an undirected graph with no simple circuits 


rooted tree: a directed graph with a specified vertex, called the 
root, such that there is a unique path to every other vertex 
from this root 


subtree: a subgraph of a tree that is also a tree 
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*24, Show that the addition of edges at each stage of Sollin’s 
algorithm produces a forest. 


25. Use Sollin’s algorithm to produce a minimum spanning 
tree for the weighted graph shown in 
a) Figure 1. 
b) Figure 3. 
* 26. Express Sollin’s algorithm in pseudocode. 


** 27, Prove that Sollin’s algorithm produces a minimum span- 


ning tree in a connected undirected weighted graph. 


* 28. Show that the first step of Sollin’s algorithm produces a 
forest containing at least [n/2] edges when the input is 
an undirected graph with n vertices. 


* 29. Show that if there are r trees in the forest at some interme- 
diate step of Sollin’s algorithm, then at least [7/2] edges 
are added by the next iteration of the algorithm. 


* 30. Show that when given as input an undirected graph with n 
vertices, no more than |n/ 2* | trees remain after the first 
step of Sollin’s algorithm has been carried out and the 
second step of the algorithm has been carried out k — 1 
times. 


* 31. Show that Sollin’s algorithm requires at most log n iter- 
ations to produce a minimum spanning tree from a con- 
nected undirected weighted graph with n vertices. 


32. Prove that Kruskal’s algorithm produces minimum span- 
ning trees. 


33. Show that if G is a weighted graph with distinct edge 
weights, then for every simple circuit of G, the edge of 
maximum weight in this circuit does not belong to any 
minimum spanning tree of G. 


When Kruskal invented the algorithm that finds minimum 
spanning trees by adding edges in order of increasing weight 
as long as they do not form a simple circuit, he also invented 
another algorithm sometimes called the reverse-delete al- 
gorithm. This algorithm proceeds by successively deleting 
edges of maximum weight from a connected graph as long as 
doing so does not disconnect the graph. 


34. Express the reverse-delete algorithm in pseudocode. 


35. Prove that the reverse-delete algorithm always produces 
a minimum spanning tree when given as input a weighted 
graph with distinct edge weights. [Hint: Use Exercise 33.] 


parent of v in a rooted tree: the vertex u such that (u, v) is an 
edge of the rooted tree 


child of a vertex v in a rooted tree: any vertex with v as its 
parent 


sibling of a vertex v in a rooted tree: a vertex with the same 
parent as v 


ancestor of a vertex v in a rooted tree: any vertex on the path 
from the root to v 
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descendant of a vertex v in a rooted tree: any vertex that has 
v as an ancestor 

internal vertex: a vertex that has children 

leaf: a vertex with no children 

level of a vertex: the length of the path from the root to this 
vertex 

height of a tree: the largest level of the vertices of a tree 

m-ary tree: a tree with the property that every internal vertex 
has no more than m children 

full m-ary tree: a tree with the property that every internal 
vertex has exactly m children 

binary tree: an m-ary tree with m = 2 (each child may be 
designated as a left or a right child of its parent) 

ordered tree: a tree in which the children of each internal 
vertex are linearly ordered 

balanced tree: a tree in which every leaf is at level h or h — 1, 
where / is the height of the tree 

binary search tree: a binary tree in which the vertices are la- 
beled with items so that a label of a vertex is greater than 
the labels of all vertices in the left subtree of this vertex and 
is less than the labels of all vertices in the right subtree of 
this vertex 

decision tree: a rooted tree where each vertex represents a 
possible outcome of a decision and the leaves represent the 
possible solutions of a problem 

game tree: a rooted tree where vertices represents the possi- 
ble positions of a game as it progresses and edges represent 
legal moves between these positions 

prefix code: a code that has the property that the code of a 
character is never a prefix of the code of another character 

minmax strategy: the strategy where the first player and sec- 
ond player move to positions represented by a child with 
maximum and minimum value, respectively 

value of a vertex in a game tree: for a leaf, the payoff to the 
first player when the game terminates in the position repre- 
sented by this leaf; for an internal vertex, the maximum or 
minimum of the values of its children, for an internal vertex 
at an even or odd level, respectively 

tree traversal: a listing of the vertices of a tree 

preorder traversal: a listing of the vertices of an ordered 
rooted tree defined recursively—the root is listed, followed 
by the first subtree, followed by the other subtrees in the 
order they occur from left to right 

inorder traversal: a listing of the vertices of an ordered rooted 
tree defined recursively—the first subtree is listed, followed 
by the root, followed by the other subtrees in the order they 
occur from left to right 


Review Questions 


b) Define a forest. 


2. Can there be two different simple paths between the ver- 
tices of a tree? 


1. a) Define a tree. 


3. Give at least three examples of how trees are used in mod- 
eling. 
4. a) Define a rooted tree and the root of such a tree. 


postorder traversal: a listing of the vertices of an ordered 
rooted tree defined recursively—the subtrees are listed in 
the order they occur from left to right, followed by the root 

infix notation: the form of an expression (including a full set 
of parentheses) obtained from an inorder traversal of the 
binary tree representing this expression 

prefix (or Polish) notation: the form of an expression ob- 
tained from a preorder traversal of the tree representing this 
expression 

postfix (or reverse Polish) notation: the form of an expression 
obtained from a postorder traversal of the tree representing 
this expression 

spanning tree: a tree containing all vertices of a graph 

minimum spanning tree: a spanning tree with smallest pos- 
sible sum of weights of its edges 


RESULTS 


A graph is a tree if and only if there is a unique simple path 
between every pair of its vertices. 

A tree with n vertices has n — 1 edges. 

A full m-ary tree with i internal vertices has mi + 1 vertices. 

The relationships among the numbers of vertices, leaves, and 
internal vertices in a full m-ary tree (see Theorem 4 in Sec- 
tion 11.1) 

There are at most m" leaves in an m-ary tree of height h. 

If an m-ary tree has / leaves, its height h is at least [log,,, /]. If 
the tree is also full and balanced, then its height is [log,,, /]. 

Huffman coding: a procedure for constructing an optimal bi- 
nary code for a set of symbols, given the frequencies of 
these symbols 

depth-first search, or backtracking: a procedure for con- 
structing a spanning tree by adding edges that form a path 
until this is not possible, and then moving back up the path 
until a vertex is found where a new path can be formed 

breadth-first search: a procedure for constructing a spanning 
tree that successively adds all edges incident to the last set 
of edges added, unless a simple circuit is formed 

Prim’s algorithm: a procedure for producing a minimum 
spanning tree in a weighted graph that successively adds 
edges with minimal weight among all edges incident to a 
vertex already in the tree so that no edge produces a simple 
circuit when it is added 

Kruskal’s algorithm: a procedure for producing a minimum 
spanning tree in a weighted graph that successively adds 
edges of least weight that are not already in the tree such 
that no edge produces a simple circuit when it is added 


b) Define the parent of a vertex and a child of a vertex in 
a rooted tree. 


c) What are an internal vertex, a leaf, and a subtree in a 
rooted tree? 


d) Draw a rooted tree with at least 10 vertices, where the 
degree of each vertex does not exceed 3. Identify the 


10. 


11. 


12. 


13. 


root, the parent of each vertex, the children of each 
vertex, the internal vertices, and the leaves. 


. a) How many edges does a tree with n vertices have? 


b) What do you need to know to determine the number 
of edges in a forest with n vertices? 


. a) Define a full m-ary tree. 


b) How many vertices does a full m-ary tree have if it 
has i internal vertices? How many leaves does the tree 
have? 


. a) What is the height of a rooted tree? 


b) What is a balanced tree? 
c) How many leaves can an m-ary tree of height h have? 


. a) What is a binary search tree? 


b) Describe an algorithm for constructing a binary search 
tree. 

c) Forma binary search tree for the words vireo, warbler, 
egret, grosbeak, nuthatch, and kingfisher. 


. a) What is a prefix code? 


b) How can a prefix code be represented by a binary tree? 


a) Define preorder, inorder, and postorder tree traversal. 

b) Give an example of preorder, postorder, and inorder 
traversal of a binary tree of your choice with at 
least 12 vertices. 


a) Explain how to use preorder, inorder, and postorder 
traversals to find the prefix, infix, and postfix forms of 
an arithmetic expression. 

b) Draw the ordered rooted tree that represents 
(x — 3) + (@&/4) + @ —y) t 3)). 

c) Find the prefix and postfix forms of the expression in 
part (b). 

Show that the number of comparisons used by a sorting 

algorithm to sort a list of n elements is at least [log n!]. 

a) Describe the Huffman coding algorithm for construct- 
ing an optimal code for a set of symbols, given the 
frequency of these symbols. 


Supplementary Exercises 


*1, 


#2, 


*6. 


Show that a simple graph is a tree if and only if it contains 
no simple circuits and the addition of an edge connecting 
two nonadjacent vertices produces a new graph that has 
exactly one simple circuit (where circuits that contain the 
same edges are not considered different). 


How many nonisomorphic rooted trees are there with six 
vertices? 


. Show that every tree with at least one edge must have at 


least two pendant vertices. 


. Show that a tree with n vertices that has n — 1 pendant 


vertices must be isomorphic to K1,,-1. 


. What is the sum of the degrees of the vertices of a tree 


with n vertices? 


Suppose that dj, d2,..., dy, are n positive integers with 
sum 2n — 2. Show that there is a tree that has n vertices 
such that the degrees of these vertices are d}, do, ..., dn. 


14. 


15. 


16. 


17. 


18. 


19. 


7. 
8. 
9. 
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b) Use Huffman coding to find an optimal code for 
these symbols and frequencies: A: 0.2, B: 0.1, C: 0.3, 
D: 0.4. 


Draw the game tree for nim if the starting position consists 
of two piles with one and four stones, respectively. Who 
wins the game if both players follow an optimal strategy? 


a) What is a spanning tree of a simple graph? 
b) Which simple graphs have spanning trees? 


c) Describe at least two different applications that require 
that a spanning tree of a simple graph be found. 


a) Describe two different algorithms for finding a span- 
ning tree in a simple graph. 

b) Illustrate how the two algorithms you described in 
part (a) can be used to find the spanning tree of a sim- 
ple graph, using a graph of your choice with at least 
eight vertices and 15 edges. 


a) Explain how backtracking can be used to determine 
whether a simple graph can be colored using n colors. 


b) Show, with an example, how backtracking can be used 
to show that a graph with a chromatic number equal 
to 4 cannot be colored with three colors, but can be 
colored with four colors. 


a) What is a minimum spanning tree of a connected 
weighted graph? 
b) Describe at least two different applications that require 


that a minimum spanning tree of aconnected weighted 
graph be found. 


a) Describe Kruskal’s algorithm and Prim’s algorithm 
for finding minimum spanning trees. 


b) Illustrate how Kruskal’s algorithm and Prim’s algo- 
rithm are used to find a minimum spanning tree, using 
a weighted graph with at least eight vertices and 15 
edges. 


Show that every tree is a planar graph. 
Show that every tree is bipartite. 


Show that every forest can be colored using two colors. 


A B-tree of degree k is a rooted tree such that all its leaves 
are at the same level, its root has at least two and at most k 
children unless it is a leaf, and every internal vertex other than 
the root has at least [k/2], but no more than k, children. Com- 
puter files can be accessed efficiently when B-trees are used 
to represent them. 


10. 
*11, 


*12, 


Draw three different B-trees of degree 3 with height 4. 


Give an upper bound and a lower bound for the number 
of leaves in a B-tree of degree k with height h. 


Give an upper bound and a lower bound for the height of 
a B-tree of degree k with n leaves. 


The binomial trees B;, i = 0,1,2,..., are ordered rooted 
trees defined recursively: 
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Basis step: The binomial tree Bo is the tree with a single 
vertex. 


Recursive step: Let k be a nonnegative integer. To con- 
struct the binomial tree By,+1, add a copy of By toa second 
copy of B; by adding an edge that makes the root of the 
first copy of B;, the leftmost child of the root of the second 
copy of Bx. 

13. Draw B; for k = 0, 1, 2,3, 4. 

14. How many vertices does B; have? Prove that your answer 
is correct. 

15. Find the height of By. Prove that your answer is correct. 

16. How many vertices are there in By at depth j, where 
0 < j < k? Justify your answer. 

17. What is the degree of the root of B,? Prove that your 
answer is correct. 

18. Show that the vertex of largest degree in B; is the root. 


A rooted tree T is called an $;,-tree if it satisfies this recursive 
definition. It is an Sg-tree if it has one vertex. For k > 0, T is 
an S;-tree if it can be built from two S,_-trees by making the 
root of one the root of the S,-tree and making the root of the 
other the child of the root of the first $,—1-tree. 


19. Draw an S;-tree for k = 0, 1, 2,3, 4. 


20. Show that an S,-tree has 2* vertices and a unique vertex 
at level k. This vertex at level k is called the handle. 


*21 


. 


Suppose that T is an S,-tree with handle v. Show that T 
can be obtained from disjoint trees 7p, 7), ..., Tk—-1, 
with roots ro, 71,...,/k—1, respectively, where v is not 
in any of these trees, where 7; is an S;-tree for i = 
0,1,..., k — 1, by connecting v to rp and r; to rj; for 
i=0,1,...,4-2. 

The listing of the vertices of an ordered rooted tree in level 
order begins with the root, followed by the vertices at level 1 
from left to right, followed by the vertices at level 2 from left 
to right, and so on. 


22. List the vertices of the ordered rooted trees in Figures 3 
and 9 of Section 11.3 in level order. 


23. Devise an algorithm for listing the vertices of an ordered 
rooted tree in level order. 


*24, Devise an algorithm for determining if a set of universal 
addresses can be the addresses of the leaves of a rooted 
tree. 


25. Devise an algorithm for constructing a rooted tree from 
the universal addresses of its leaves. 


A cut set of a graph is a set of edges such that the removal of 
these edges produces a subgraph with more connected com- 
ponents than in the original graph, but no proper subset of this 
set of edges has this property. 


26. Show that a cut set of a graph must have at least one edge 
in common with any spanning tree of this graph. 


A cactus is a connected graph in which no edge is in more 
than one simple circuit not passing through any vertex other 
than its initial vertex more than once or its initial vertex other 
than at its terminal vertex (where two circuits that contain the 
same edges are not considered different). 


27. Which of these graphs are cacti? 
a) 


28. Is a tree necessarily a cactus? 


29. Show that a cactus is formed if we add a circuit containing 
new edges beginning and ending at a vertex of a tree. 


* 30. Show that if every circuit not passing through any vertex 
other than its initial vertex more than once in a connected 
graph contains an odd number of edges, then this graph 
must be a cactus. 


A degree-constrained spanning tree of a simple graph G 
is a spanning tree with the property that the degree of a ver- 
tex in this tree cannot exceed some specified bound. Degree- 
constrained spanning trees are useful in models of transporta- 
tion systems where the number of roads at an intersection is 
limited, models of communications networks where the num- 
ber of links entering a node is limited, and so on. 

In Exercises 31-33 find a degree-constrained spanning 
tree of the given graph where each vertex has degree less than 
or equal to 3, or show that such a spanning tree does not exist. 


31. 32. b 


QQ 
mw 


33. a b c 


i h g 
34. Show that a degree-constrained spanning tree of a simple 
graph in which each vertex has degree not exceeding 2 
consists of a single Hamilton path in the graph. 


35. A tree with n vertices is called graceful if its vertices 
can be labeled with the integers 1, 2, ..., such that the 
absolute values of the difference of the labels of adja- 
cent vertices are all different. Show that these trees are 
graceful. 

a a 


A caterpillar is a tree that contains a simple path such that 
every vertex not contained in this path is adjacent to a vertex 
in the path. 


36. Which of the graphs in Exercise 35 are caterpillars? 


37. How many nonisomorphic caterpillars are there with six 
vertices? 


** 38, a) Prove or disprove that all trees whose edges form a 
single path are graceful. 
b) Prove or disprove that all caterpillars are graceful. 


39. Suppose that in a long bit string the frequency of occur- 
rence of a 0 bit is 0.9 and the frequency of a 1 bit is 0.1 
and bits occur independently. 

a) Construct a Huffman code for the four blocks of two 
bits, 00, 01, 10, and 11. What is the average number 
of bits required to encode a bit string using this code? 

b) Construct a Huffman code for the eight blocks of three 
bits. What is the average number of bits required to en- 
code a bit string using this code? 

40. Suppose that G is a directed graph with no circuits. De- 
scribe how depth-first search can be used to carry out a 
topological sort of the vertices of G. 


*41. Suppose that e is an edge in a weighted graph that is in- 
cident to a vertex v such that the weight of e does not 
exceed the weight of any other edge incident to v. Show 
that there exists a minimum spanning tree containing this 
edge. 

42. Three couples arrive at the bank of a river. Each of the 
wives is jealous and does not trust her husband when he 
is with one of the other wives (and perhaps with other 
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people), but not with her. How can six people cross to the 
other side of the river using a boat that can hold no more 
than two people so that no husband is alone with a woman 
other than his wife? Use a graph theory model. 

* 43, Show that if no two edges in a weighted graph have the 
same weight, then the edge with least weight incident to 
a vertex v is included in every minimum spanning tree. 


44, Find a minimum spanning tree of each of these graphs 
where the degree of each vertex in the spanning tree does 
not exceed 2. 


Let G = (V, E) bea directed graph and let r be a vertex in G. 
An arborescence of G rooted at r is a subgraph T = (V, F) 
of G such that the underlying undirected graph of T is a span- 
ning tree of the underlying undirected graph of G and for every 
vertex v € V there is a path from r to v in T (with directions 
taken into account). 


45. Show that a subgraph T = (V, F) of the graph G = 
(V, E) is an arborescence of G rooted at r if and only 
if T contains r, T has no simple circuits, and for every 
vertex v € V other than, deg’ (v) = 1inT. 


46. Show that a directed graph G = (V, E) has an arbores- 
cence rooted at the vertex r if and only if for every vertex 
v € V, there is a directed path from r to v. 


47. In this exercise we will develop an algorithm to find the 
strong components of a directed graph G = (V, E). Re- 
call that a vertex w € V is reachable from a vertex v € V 
if there is a directed path from v to w. 

a) Explain how to use breadth-first search in the directed 
graph G to find all the vertices reachable from a vertex 
veG. 

b) Explain how to use breadth-first search in G°”” to find 
all the vertices from which a vertex v € G is reachable. 
(Recall that G°”” is the directed graph obtained from 
G by reversing the direction of all its edges.) 

c) Explain how to use parts (a) and (b) to construct an al- 
gorithm that finds the strong components of a directed 
graph G, and explain why your algorithm is correct. 
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Computer Projects 


Write programs with these input and output. 


10. 


. Given the adjacency matrix of an undirected simple graph, 


determine whether the graph is a tree. 


. Given the adjacency matrix of a rooted tree and a vertex in 


the tree, find the parent, children, ancestors, descendants, 
and level of this vertex. 


. Given the list of edges of a rooted tree and a vertex in the 


tree, find the parent, children, ancestors, descendants, and 
level of this vertex. 


. Given a list of items, construct a binary search tree con- 


taining these items. 


. Given a binary search tree and an item, locate or add this 


item to the binary search tree. 


. Given the ordered list of edges of an ordered rooted tree, 


find the universal addresses of its vertices. 


. Given the ordered list of edges of an ordered rooted tree, 


list its vertices in preorder, inorder, and postorder. 


. Given an arithmetic expression in prefix form, find its 


value. 


. Given an arithmetic expression in postfix form, find its 


value. 


Given the frequency of symbols, use Huffman coding to 
find an optimal code for these symbols. 


Computations and Explorations 


11. 


12. 


13. 


14. 


*15, 


*16. 


17. 


18. 


Given an initial position in the game of nim, determine 
an optimal strategy for the first player. 


Given the adjacency matrix of a connected undirected 
simple graph, find a spanning tree for this graph using 
depth-first search. 

Given the adjacency matrix of a connected undirected 
simple graph, find a spanning tree for this graph using 
breadth-first search. 


Given a set of positive integers and a positive integer N, 
use backtracking to find a subset of these integers that 
have WN as their sum. 


Given the adjacency matrix of an undirected simple graph, 
use backtracking to color the graph with three colors, if 
this is possible. 


Given a positive integer n, solve the n-queens problem 
using backtracking. 


Given the list of edges and their weights of a weighted 
undirected connected graph, use Prim’s algorithm to find 
a minimum spanning tree of this graph. 

Given the list of edges and their weights of a weighted 
undirected connected graph, use Kruskal’s algorithm to 
find a minimum spanning tree of this graph. 


Use a computational program or programs you have written to do these exercises. 


1. Display all trees with six vertices. 


#3, 


. Display a full set of nonisomorphic trees with seven ver- 


tices. 


Construct a Huffman code for the symbols with ASCII 
codes given the frequency of their occurrence in represen- 
tative input. 


. Compute the number of different spanning trees of K,, for 


n = 1, 2,3,4,5, 6. Conjecture a formula for the number 
of such spanning trees whenever n is a positive integer. 


. Compare the number of comparisons needed to sort lists of 


n elements for n = 100, 1000, and 10,000 from the set of 
positive integers less than 1,000,000, where the elements 


Writing Projects 


Respond to these with essays using outside sources. 


. Explain how Cayley used trees to enumerate the number 


of certain types of hydrocarbons. 


. Explain how trees are used to represent ancestral relations 


in the study of evolution. 


#7. 


3. 


are randomly selected positive integers, using the selec- 
tion sort, the insertion sort, the merge sort, and the quick 
sort. 


. Compute the number of different ways n queens can be 


arranged on an n x n chessboard so that no two queens 
can attack each other for all positive integers n not 
exceeding 10. 


Find a minimum spanning tree of the graph that connects 
the capital cities of the 50 states in the United States to 
each other where the weight of each edge is the distance 
between the cities. 


. Draw the complete game tree for a game of checkers on a 


4 x 4 board. 


Discuss hierarchical cluster trees and how they are used. 


. Define AVL-trees (sometimes also known as height- 


balanced trees). Describe how and why AVL-trees are 
used in a variety of different algorithms. 


10. 


. Define quad trees and explain how images can be repre- 


sented using them. Describe how images can be rotated, 
scaled, and translated by manipulating the corresponding 
quad tree. 


. Define a heap and explain how trees can be turned into 


heaps. Why are heaps useful in sorting? 


. Describe dynamic algorithms for data compression based 


on letter frequencies as they change as characters are suc- 
cessively read, such as adaptive Huffman coding. 


. Explain how alpha-beta pruning can be used to simplify 


the computation of the value of a game tree. 


. Describe the techniques used by chess-playing programs 


such as Deep Blue. 


Define the type of graph known as a mesh of trees. Ex- 
plain how this graph is used in applications to very large 
system integration and parallel computing. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 
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Discuss the algorithms used in IP multicasting to avoid 
loops between routers. 

Describe an algorithm based on depth-first search for find- 
ing the articulation points of a graph. 

Describe an algorithm based on depth-first search to find 
the strongly connected components of a directed graph. 
Describe the search techniques used by the crawlers and 
spiders in different search engines on the Web. 

Describe an algorithm for finding the minimum spanning 
tree of a graph such that the maximum degree of any ver- 
tex in the spanning tree does not exceed a fixed constant k. 
Compare and contrast some of the most important sorting 
algorithms in terms of their complexity and when they are 
used. 

Discuss the history and origins of algorithms for con- 
structing minimum spanning trees. 


Describe algorithms for producing random trees. 


12.1 Boolean 
Functions 


12.2 Representing 
Boolean 
Functions 


12.3 Logic Gates 


12.4 Minimization 
of Circuits 


Boolean Algebra 


T he circuits in computers and other electronic devices have inputs, each of which is either 
a0 ora 1, and produce outputs that are also Os and Is. Circuits can be constructed using 
any basic element that has two different states. Such elements include switches that can be 
in either the on or the off position and optical devices that can be either lit or unlit. In 1938 
Claude Shannon showed how the basic rules of logic, first given by George Boole in 1854 in his 
The Laws of Thought, could be used to design circuits. These rules form the basis for Boolean 
algebra. In this chapter we develop the basic properties of Boolean algebra. The operation of a 
circuit is defined by a Boolean function that specifies the value of an output for each set of inputs. 
The first step in constructing a circuit is to represent its Boolean function by an expression built 
up using the basic operations of Boolean algebra. We will provide an algorithm for producing 
such expressions. The expression that we obtain may contain many more operations than are 
necessary to represent the function. Later in the chapter we will describe methods for finding an 
expression with the minimum number of sums and products that represents a Boolean function. 
The procedures that we will develop, Karnaugh maps and the Quine—McCluskey method, are 
important in the design of efficient circuits. 


Boolean Functions 


EXAMPLE 1 


Introduction 


Boolean algebra provides the operations and the rules for working with the set {0, 1}. Electronic 
and optical switches can be studied using this set and the rules of Boolean algebra. The three 
operations in Boolean algebra that we will use most are complementation, the Boolean sum, and 
the Boolean product. The complement of an element, denoted with a bar, is defined by 0 = 1 
and 1 = 0. The Boolean sum, denoted by + or by OR, has the following values: 


14+1=1, 1+0=1, O+1=1, 0+0=0. 
The Boolean product, denoted by - or by AND, has the following values: 

1-1=1, 1-0=0, 0-1=0, 0-0=0. 
When there is no danger of confusion, the symbol - can be deleted, just as in writing algebraic 
products. Unless parentheses are used, the rules of precedence for Boolean operators are: first, 


all complements are computed, followed by all Boolean products, followed by all Boolean sums. 
This is illustrated in Example 1. 


Find the value of 1-0+ (0+ 1). 


Solution: Using the definitions of complementation, the Boolean sum, and the Boolean product, 
it follows that 
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EXAMPLE 2 


EXAMPLE 3 


The complement, Boolean sum, and Boolean product correspond to the logical operators, 
=, V, and A, respectively, where 0 corresponds to F (false) and 1 corresponds to T (true). Equal- 
ities in Boolean algebra can be directly translated into equivalences of compound propositions. 
Conversely, equivalences of compound propositions can be translated into equalities in Boolean 
algebra. We will see later in this section why these translations yield valid logical equivalences 
and identities in Boolean algebra. Example 2 illustrates the translation from Boolean algebra to 
propositional logic. 


Translate 1 -0-+ (0 + 1) = 0, the equality found in Example 1, into a logical equivalence. 
Solution: We obtain a logical equivalence when we translate each | into a T, each 0 into 


an F, each Boolean sum into a disjunction, each Boolean product into a conjunction, and each 
complementation into a negation. We obtain 


(TAF)v-(Tv F) =F. > | 


Example 3 illustrates the translation from propositional logic to Boolean algebra. 


Translate the logical equivalence (T A T) V =F = T into an identity in Boolean algebra. 


Solution: We obtain an identity in Boolean algebra when we translate each T into a 1, each F 
into a O, each disjunction into a Boolean sum, each conjunction into a Boolean product, and 
each negation into a complementation. We obtain 


(-1)+0=1. “ 


Boolean Expressions and Boolean Functions 


Let B = {0, 1}. Then B” = {(x1, x2,...,Xn) | x; € B for 1 <i <n} is the set of all possible 
n-tuples of Os and 1s. The variable x is called a Boolean variable if it assumes values only 
from B, that is, if its only possible values are 0 and 1. A function from B” to B is called a 
Boolean function of degree n. 


gy CLAUDE ELWOOD SHANNON (1916-2001) Claude Shannon was born in Petoskey, Michigan, and grew up 


in Gaylord, Michigan. His father was a businessman and a probate judge, and his mother was a language teacher 
and a high school principal. Shannon attended the University of Michigan, graduating in 1936. He continued 
his studies at M.LT., where he took the job of maintaining the differential analyzer, a mechanical computing 
device consisting of shafts and gears built by his professor, Vannevar Bush. Shannon’s master’s thesis, written 
in 1936, studied the logical aspects of the differential analyzer. This master’s thesis presents the first application 
of Boolean algebra to the design of switching circuits; it is perhaps the most famous master’s thesis of the 
twentieth century. He received his Ph.D. from M.LT. in 1940. Shannon joined Bell Laboratories in 1940, where 
he worked on transmitting data efficiently. He was one of the first people to use bits to represent information. At 


Bell Laboratories he worked on determining the amount of traffic that telephone lines can carry. Shannon made many fundamental 
contributions to information theory. In the early 1950s he was one of the founders of the study of artificial intelligence. He joined 
the M.LT. faculty in 1956, where he continued his study of information theory. 

Shannon had an unconventional side. He is credited with inventing the rocket-powered Frisbee. He is also famous for riding a 
unicycle down the hallways of Bell Laboratories while juggling four balls. Shannon retired when he was 50 years old, publishing 
papers sporadically over the following 10 years. In his later years he concentrated on some pet projects, such as building a motorized 
pogo stick. One interesting quote from Shannon, published in Omni Magazine in 1987, is “I visualize a time when we will be to 
robots what dogs are to humans. And I am rooting for the machines.” 


12.1 Boolean Functions 813 


EXAMPLE 4 The function F(x, y) = xy from the set of ordered pairs of Boolean variables to the set {0, 1} is 


EXAMPLE 5 


EXAMPLE 6 


110 
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FIGURE 1 
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a Boolean function of degree 2 with F(1, 1) = 0, FU, 0) = 1, F(O, 1) = 0, and F(0, 0) = 0. 
We display these values of F in Table 1. < 


Boolean functions can be represented using expressions made up from variables and Boolean 


operations. The Boolean expressions in the variables x), x2, ..., x, are defined recursively as 
0, 1, x1, .x2,..., X, are Boolean expressions; _ 
if E, and E> are Boolean expressions, then F;, (E; £2), and (E; + E2) are Boolean ex- 
pressions. 


Each Boolean expression represents a Boolean function. The values of this function are obtained 
by substituting 0 and | for the variables in the expression. In Section 12.2 we will show that 
every Boolean function can be represented by a Boolean expression. 


Find the values of the Boolean function represented by F(x, y, z) = xy + Z. 


Solution: The values of this function are displayed in Table 2. < 


TABLE 2 


> [2 | [2 | Ponca 


1 
1 
1 
1 
0 
0 
0 
0 


GCaOorroor.r 
or OF OF OF 
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Note that we can represent a Boolean function graphically by distinguishing the vertices of 
the n-cube that correspond to the n-tuples of bits where the function has value 1. 


The function F(x, y, z) = xy +Z from B? to B from Example 5 can be represented by distin- 
guishing the vertices that correspond to the five 3-tuples (1, 1, 1), (1, 1, 0), C1, 0, 0), (0, 1, 0), 
and (0, 0,0), where F(x, y, z) = 1, as shown in Figure |. These vertices are displayed using 
solid black circles. 


Boolean functions F and G of n variables are equal if and only if F(b,, b2,..., bn) = 
G(by, b2,..., bn) whenever by, b2, ..., by) belong to B. Two different Boolean expressions that 
represent the same function are called equivalent. For instance, the Boolean expressions xy, 
xy +0, and xy - | are equivalent. The complement of the Boolean function F is the function F, 
where F(x1,...,X,) = F(x1,..., Xn). Let F and G be Boolean functions of degree n. The 
Boolean sum F + G and the Boolean product FG are defined by 


(F + G)(x1,...5 Xn) = F(x, vis ha) F OCig0.s5 ta) 
(FG)(x1,...,%n) = F(q,...,Xn)G(X1,...,Xn)- 


A Boolean function of degree two is a function from a set with four elements, namely, 
pairs of elements from B = {0, 1}, to B, a set with two elements. Hence, there are 16 different 
Boolean functions of degree two. In Table 3 we display the values of the 16 different Boolean 
functions of degree two, labeled F), F2,..., Fie. 
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TABLE 3 The 16 Boolean Functions of Degree Two. 


1] 1 1 1 


oor 
ooo 


EXAMPLE 7 


EXAMPLE 8 


1 if 1 1 1 
1 
0 
1 


How many different Boolean functions of degree n are there? 


Solution: From the product rule for counting, it follows that there are 2” different n-tuples 
of Os and 1s. Because a Boolean function is an assignment of 0 or | to each of these 2” different 
n-tuples, the product rule shows that there are 27" different Boolean functions of degreen. <4 


Table 4 displays the number of different Boolean functions of degrees one through six. The 
number of such functions grows extremely rapidly. 


TABLE 4 The Number of Boolean 
Functions of Degree n. 


256 

65,536 

4,294,967,296 
18,446,744,073,709,551,616 


Identities of Boolean Algebra 


There are many identities in Boolean algebra. The most important of these are displayed in 
Table 5. These identities are particularly useful in simplifying the design of circuits. Each of 
the identities in Table 5 can be proved using a table. We will prove one of the distributive laws 
in this way in Example 8. The proofs of the remaining properties are left as exercises for the 
reader. 


Show that the distributive law x(y + z) = xy + xz is valid. 


Solution: The verification of this identity is shown in Table 6. The identity holds because the 
last two columns of the table agree. < 


The reader should compare the Boolean identities in Table 5 to the logical equivalences 
in Table 6 of Section 1.3 and the set identities in Table 1 in Section 2.2. All are special cases 
of the same set of identities in a more abstract structure. Each collection of identities can 
be obtained by making the appropriate translations. For example, we can transform each of 
the identities in Table 5 into a logical equivalence by changing each Boolean variable into a 
propositional variable, each 0 into a F, each | into a T, each Boolean sum into a disjunction, each 
Boolean product into a conjunction, and each complementation into a negation, as we illustrate 
in Example 9. 


Compare these Boolean 
identities with the logical 
equivalences in Section 
1.3 and the set identities 
in Section 2.2. 
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TABLE 5 Boolean Identities. 
Law of the double complement 


QY+tz2=(@+y)4 Associative laws 


Zz 
x+yz=(x+y)% +2) Distributive laws 
x(y+z)=xy+xz 


De Morgan’s laws 


Absorption laws 


Unit property 
Zero property 


EXAMPLE 9 Translate the distributive law x + yz = (x + y)(x + z) in Table 5 into a logical equivalence. 


Solution: To translate a Boolean identity into a logical equivalence, we change each Boolean 
variable into a propositional variable. Here we will change the Boolean variables x, y, and z into 
the propositional variables p, g, andr. Next, we change each Boolean sum into a disjunction and 
each Boolean product into a conjunction. (Note that 0 and | do not appear in this identity and 


TABLE 6 Verifying One of the Distributive Laws. 


ees 
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1 
0 
0 
1 
1 
0 
0 


oO = oOo =] oo => co = 


816 


12 / Boolean Algebra 


EXAMPLE 10 


Extra p> 
Examples a 


EXAMPLE 11 


EXAMPLE 12 


complementation also does not appear.) This transforms the Boolean identity into the logical 
equivalence 


PVGAT)=(PYQAMYV?). 


This logical equivalence is one of the distributive laws for propositional logic in Table 6 in 
Section 1.3. < 


Identities in Boolean algebra can be used to prove further identities. We demonstrate this 
in Example 10. 


Prove the absorption law x(x + y) = x using the other identities of Boolean algebra shown in 
Table 5. (This is called an absorption law because absorbing x + y into x leaves x unchanged.) 


Solution: We display steps used to derive this identity and the law used in each step: 


x(x + y) = (x«+0)(+ y) Identity law for the Boolean sum 


=x+0- y Distributive law of the Boolean sum over the 
Boolean product 
=xt+y- 0 Commutative law for the Boolean product 
=x+0 Domination law for the Boolean product 
=x Identity law for the Boolean sum. < 


Duality 


The identities in Table 5 come in pairs (except for the law of the double complement and the unit 
and zero properties). To explain the relationship between the two identities in each pair we use 
the concept of a dual. The dual of a Boolean expression is obtained by interchanging Boolean 
sums and Boolean products and interchanging Os and Is. 


Find the duals of x(y + 0) andx-1+ (+2). 


Solution: Interchanging - signs and + signs and interchanging Os and Is in these expressions 
produces their duals. The duals are x + (y - 1) and (x + 0)(yz), respectively. 


The dual of a Boolean function F' represented by a Boolean expression is the function 
represented by the dual of this expression. This dual function, denoted by F“, does not depend on 
the particular Boolean expression used to represent F. Anidentity between functions represented 
by Boolean expressions remains valid when the duals of both sides of the identity are taken. 
(See Exercise 30 for the reason why this is true.) This result, called the duality principle, is 
useful for obtaining new identities. 


Construct an identity from the absorption law x(x + y) = x by taking duals. 


Solution: Taking the duals of both sides of this identity produces the identity x + xy = x, which 
is also called an absorption law and is shown in Table 5. 4 


DEFINITION 1 
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The Abstract Definition of a Boolean Algebra 


In this section we have focused on Boolean functions and expressions. However, the results we 
have established can be translated into results about propositions or results about sets. Because 
of this, it is useful to define Boolean algebras abstractly. Once it is shown that a particular 
structure is a Boolean algebra, then all results established about Boolean algebras in general 
apply to this particular structure. 

Boolean algebras can be defined in several ways. The most common way is to specify the 
properties that operations must satisfy, as is done in Definition 1. 


A Boolean algebra is a set B with two binary operations V and A, elements 0 and 1, and a 
unary operation ~ such that these properties hold for all x, y, and z in B: 


xX V 0 eK ; 

eK = "| Identity laws 

56 YF = IL 

LA = il Complement laws 


Associative laws 


XVy)VZ=XV(YVVZ) 
(XAY)JAZ=XA(VAZ) 
ee 


Commutative laws 
NY SV Not 


xV(YAZ=HAVYAAV | Distributive laws 


XA(YVZ)=HAY)V (AZ) 


Using the laws given in Definition 1, it is possible to prove many other laws that hold for every 
Boolean algebra, such as idempotent and domination laws. (See Exercises 35-42.) 

From our previous discussion, B = {0, 1} with the OR and AND operations and the com- 
plement operator, satisfies all these properties. The set of propositions in n variables, with the Vv 
and A operators, F and T, and the negation operator, also satisfies all the properties of a Boolean 
algebra, as can be seen from Table 6 in Section 1.3. Similarly, the set of subsets of a universal 
set U with the union and intersection operations, the empty set and the universal set, and the 
set complementation operator, is a Boolean algebra as can be seen by consulting Table | in 
Section 2.2. So, to establish results about each of Boolean expressions, propositions, and sets, 
we need only prove results about abstract Boolean algebras. 

Boolean algebras may also be defined using the notion of a lattice, discussed in Chapter 9. 
Recall that a lattice L is a partially ordered set in which every pair of elements x, y has a least 
upper bound, denoted by lub(x, y) and a greatest lower bound denoted by glb(x, y). Given two 
elements x and y of L, we can define two operations V and A on pairs of elements of L by 
x V y = lub(x, y) andx A y = glb(x, y). 

For a lattice L to bea Boolean algebra as specified in Definition 1, it must have two properties. 
First, it must be complemented. For a lattice to be complemented it must have a least element 0 
and a greatest element | and for every element x of the lattice there must exist an element x such 
that x v x = | and x A x = 0. Second, it must be distributive. This means that for every x, y, 
and zin L,xV(yAz)=(*Vy)A(X Vz) and xA (VV Zz) = (MAY) V (XA Zz). Showing 
that a complemented, distributive lattice is a Boolean algebra has been left as Supplementary 
Exercise 39 in Chapter 9. 
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Exercises 
1. Find the values of these expressions. 
a) 1-0 b) 1+1 c) 0-0 d) (1+ 0) 
2. Find the values, if any, of the Boolean variable x that 


satisfy these equations. 
a) x-1=0 
c) x-l=x 


b) x+x=0 
d) x-x=1 


. a) Show that (1-1) + 0-1+0) =1. 


b) Translate the equation in part (a) into a propositional 
equivalence by changing each 0 into an F, each 1 
into a T, each Boolean sum into a disjunction, each 
Boolean product into a conjunction, each complemen- 
tation into a negation, and the equals sign into a propo- 
sitional equivalence sign. 

a) Show that (1-0) + (1-0) = 1. 

b) Translate the equation in part (a) into a propositional 
equivalence by changing each 0 into an F, each 1 
into a T, each Boolean sum into a disjunction, each 
Boolean product into a conjunction, each complemen- 
tation into a negation, and the equals sign into a propo- 
sitional equivalence sign. 


. Use a table to express the values of each of these Boolean 


functions. 

a) F(x, y,z) =Xy 

b) F(x, y,z)=x+ yz 

ce) F(x, y,z) =xy + (xyz) 
d) F(x, y,z) =x(yz+yZ) 


. Use a table to express the values of each of these Boolean 


functions. 

a) F(X, y,z)=Z 

b) F(x, y,z) =xXy+ yz 

c) F(x, y,z) = xyz + (xyz) 
d) F(x, y,z) = yxz+%XZ) 


. Use a 3-cube Q3 to represent each of the Boolean func- 


tions in Exercise 5 by displaying a black circle at each 
vertex that corresponds to a 3-tuple where this function 
has the value 1. 


. Use a 3-cube Q3 to represent each of the Boolean func- 


tions in Exercise 6 by displaying a black circle at each 
vertex that corresponds to a 3-tuple where this function 
has the value 1. 


9. What values of the Boolean variables x and y satisfy 
xy=x+y? 
10. How many different Boolean functions are there of de- 
gree 7? 
11. Prove the absorption law x + xy = x using the other laws 
in Table 5. 
Es" 12. Show that F(x, y, z) = xy + xz + yz has the value 1 if 


13. 


and only if at least two of the variables x, y, and z have 
the value 1. 


Show that xy + yZ+xz=Xy+yz4+ xz. 


Exercises 14—23 deal with the Boolean algebra {0, 1} with ad- 
dition, multiplication, and complement defined at the begin- 
ning of this section. In each case, use a table as in Example 8. 


14. 
15. 
16. 
17. 
18. 
19. 
20. 
21. 
22. 
23. 


Verify the law of the double complement. 
Verify the idempotent laws. 

Verify the identity laws. 

Verify the domination laws. 

Verify the commutative laws. 

Verify the associative laws. 

Verify the first distributive law in Table 5. 
Verify De Morgan’s laws. 

Verify the unit property. 

Verify the zero property. 


The Boolean operator @, called the XOR operator, is defined 
by161=0,160=1,061=1,and0G60=0. 


24. 


25. 


26. 
27. 


28. 


* 29, 


* 30. 


*31. 


* 32. 


33. 


34. 


Simplify these expressions. 

a) x@0 b) x@l 
c) x@x d) x@®x 
Show that these identities hold. 


a) x@y=(x+ yay) 
b) x ®y = (xy) + Gy) 


Show thatx @y=y@x. 

Prove or disprove these equalities. 

a) x8 O27=A8y) Gz 

bh x+y O2=At+y) OW+2z) 


ce) xO(¥+2=%Oy)+0 82) 

Find the duals of these Boolean expressions. 

a) x+y b) xy 

ce) xyzt+xYZ d) xz+x-04+%-1 
Suppose that F is a Boolean function represented by a 
Boolean expression in the variables x1, ... , X,. Show that 
Po Sis én5 Mp) = Fee 

Show that if F and G are Boolean functions represented 
by Boolean expressions in n variables and F = G, then 
Fl= G4, where F@ and G@ are the Boolean functions 
represented by the duals of the Boolean expressions rep- 
resenting F and G, respectively. [Hint: Use the result of 
Exercise 29.] 

How many different Boolean functions F(x, y, z) are 
there such that F(x, y, Zz) = F(x, y, z) for all values of 
the Boolean variables x, y, and z? 

How many different Boolean functions F(x, y, z) are 
there such that F(X, y, z) = F(x, y, z) = F(x, y,Z) for 
all values of the Boolean variables x, y, and z? 

Show that you obtain De Morgan’s laws for propositions 
(in Table 6 in Section 1.3) when you transform De Mor- 
gan’s laws for Boolean algebra in Table 6 into logical 
equivalences. 

Show that you obtain the absorption laws for proposi- 
tions (in Table 6 in Section 1.3) when you transform the 
absorption laws for Boolean algebra in Table 6 into logi- 
cal equivalences. 
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In Exercises 35—42, use the laws in Definition 1 to show that That is, show that for all x and y, (x V y) =X A y and 
the stated properties hold in every Boolean algebra. (xAy)=XVY. 


40. Show that in a Boolean algebra, the modular properties 
hold. That is, show that x A (vy V (x Az)) = (xA y) Vv 
(xAzyandx V(VA(XVZ))=(&VY)AMV2Z). 


35. Show that in a Boolean algebra, the idempotent laws 
x Vx =x andx Ax = x hold for every element x. 


36. Show that in a Boolean algebra, every element « has a 41. Show that in a Boolean algebra, if x V y = 0, thenx = 0 
unique complement suchthatx v ¥ = landx Ax =0. and y = 0, and that ifx A y = 1, thenx = land y = 1. 
37. Show that in a Boolean algebra, the complement of the 42. Show that in a Boolean algebra, the dual of an iden- 
element 0 is the element 1 and vice versa. tity, obtained by interchanging the V and A operators 
; and interchanging the elements 0 and 1, is also a valid 

38. Prove that in a Boolean algebra, the law of the double identity 

1 t holds; that is, x = x for 1 tx. : 
ii ca faa a oi eae 43. Show that a complemented, distributive lattice is a 

39. Show that De Morgan’s laws hold in a Boolean algebra. Boolean algebra. 


Representing Boolean Functions 


Two important problems of Boolean algebra will be studied in this section. The first problem 
is: Given the values of a Boolean function, how can a Boolean expression that represents this 
function be found? This problem will be solved by showing that any Boolean function can be 
represented by a Boolean sum of Boolean products of the variables and their complements. The 
solution of this problem shows that every Boolean function can be represented using the three 
Boolean operators -, +, and ~. The second problem is: Is there a smaller set of operators that 
can be used to represent all Boolean functions? We will answer this question by showing that 
all Boolean functions can be represented using only one operator. Both of these problems have 
practical importance in circuit design. 


Sum-of-Products Expansions 


We will use examples to illustrate one important way to find a Boolean expression that represents 
a Boolean function. 


EXAMPLE 1 _ Find Boolean expressions that represent the functions F(x, y, z) and G(x, y, z), which are given 
in Table 1. 


Solution: An expression that has the value 1 when x = z = | and y = 0, and the value 0 other- 
wise, is needed to represent F’. Such an expression can be formed by taking the Boolean product 
of x, y, and z. This product, xyz, has the value | if and only if x = y = z = 1, which holds if 
and only if x = z= l andy = 0. 

To represent G, we need an expression that equals | whenx = y = landz =0,orx =z= 
0 and y = 1. We can form an expression with these values by taking the Boolean sum of two 
different Boolean products. The Boolean product xyz has the value 1 if and only if x = y = 1 
and z = 0. Similarly, the product xyz has the value | if and only if x = z = 0 and y = 1. The 
Boolean sum of these two products, xyz + X yz, represents G, because it has the value 1 if and 
only ifx = y= 1landz=0,orx =z=Oandy=1. < 


= 
> 
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ee 
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Example | illustrates a procedure for constructing a Boolean expression representing a 
function with given values. Each combination of values of the variables for which the function 
has the value 1 leads to a Boolean product of the variables or their complements. 


1 
0 
1 
0 
i 
0 
1 
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DEFINITION 1 


EXAMPLE 2 


Links 


EXAMPLE 3 


Extra 
Examples 


~@ 


A literal is a Boolean variable or its complement. A minterm of the Boolean variables 
X1,X2,...,%X, iS a Boolean product y; y2--- y,, where yj = x; or yj = x;. Hence, a minterm 
is a product of v literals, with one literal for each variable. 


A minterm has the value | for one and only one combination of values of its variables. More 
precisely, the minterm y;y2...y, is 1 if and only if each y; is 1, and this occurs if and only 
if x; = | when y; = x; and x; = O when y; = X;. 


Find a minterm that equals | if x; = x3 = 0 and x2 = x4 = x5 = 1, and equals 0 otherwise. 


Solution: The minterm xX 1.x2x*3x4x5 has the correct set of values. < 


By taking Boolean sums of distinct minterms we can build up a Boolean expression with a 
specified set of values. In particular, a Boolean sum of minterms has the value | when exactly 
one of the minterms in the sum has the value 1. It has the value 0 for all other combinations of 
values of the variables. Consequently, given a Boolean function, a Boolean sum of minterms 
can be formed that has the value 1 when this Boolean function has the value 1, and has the 
value 0 when the function has the value 0. The minterms in this Boolean sum correspond to 
those combinations of values for which the function has the value 1. The sum of minterms that 
represents the function is called the sum-of-products expansion or the disjunctive normal 
form of the Boolean function. 

(See Exercise 42 in Section 1.3 for the development of disjunctive normal form in propo- 
sitional calculus.) 


Find the sum-of-products expansion for the function F(x, y, z) = (x + y)z. 


Solution: We will find the sum-of-products expansion of F(x, y, z) in two ways. First, we will 
use Boolean identities to expand the product and simplify. We find that 


F(x, y,z)=(x+ y)z 
=xz+ yz Distributive law 
= x1zZ+ lyz Identity law 
=x(y+y)z+(xat+x)yz Unit property 


=XYZ+XVZ+xXYZ+Xyz Distributive law 
= XYZ +XVZ+ XYZ. Idempotent law 


Second, we can construct the sum-of-products expansion by determining the values of F for 
all possible values of the variables x, y, and z. These values are found in Table 2. The sum-of- 
products expansion of F is the Boolean sum of three minterms corresponding to the three rows 
of this table that give the value | for the function. This gives 


F(x, y,Z) =xyZ+xyZ+ Xyz. 4 


It is also possible to find a Boolean expression that represents a Boolean function by taking 
a Boolean product of Boolean sums. The resulting expansion is called the conjunctive normal 
form or product-of-sums expansion of the function. These expansions can be found from 
sum-of-products expansions by taking duals. How to find such expansions directly is described 
in Exercise 10. 
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Functional Completeness 


Every Boolean function can be expressed as a Boolean sum of minterms. Each minterm is the 
Boolean product of Boolean variables or their complements. This shows that every Boolean 
function can be represented using the Boolean operators -, +, and ~. Because every Boolean 
function can be represented using these operators we say that the set {-, +,~ } is functionally 
complete. Can we find a smaller set of functionally complete operators? We can do so if one 
of the three operators of this set can be expressed in terms of the other two. This can be done 
using one of De Morgan’s laws. We can eliminate all Boolean sums using the identity 


x+y=xXy, 


which is obtained by taking complements of both sides in the second De Morgan law, given in 
Table 5 in Section 12.1, and then applying the double complementation law. This means that 
the set {-,~ } is functionally complete. Similarly, we could eliminate all Boolean products using 
the identity 


xy=x+y, 


which is obtained by taking complements of both sides in the first De Morgan law, given in 
Table 5 in Section 12.1, and then applying the double complementation law. Consequently 
{+,~ } is functionally complete. Note that the set {+,-} is not functionally complete, be- 
cause it is impossible to express the Boolean function F(x) = x using these operators (see 
Exercise 19). 

We have found sets containing two operators that are functionally complete. Can we find 
a smaller set of functionally complete operators, namely, a set containing just one opera- 
tor? Such sets exist. Define two operators, the | or NAND operator, defined by | | 1 = 0 and 
1|0=0|1=0]|0=1; and the | or NOR operator, defined by 1 | 1=1)/0=0)/1=0 
and 0 | 0 = 1. Both of the sets {|} and { | } are functionally complete. To see that {|} is 
functionally complete, because {-,~ } is functionally complete, all that we have to do is show 
that both of the operators - and ~ can be expressed using just the | operator. This can be done as 


X =X | xs 


xy =(x| y)| @ | y). 


The reader should verify these identities (see Exercise 14). We leave the demonstration that { | } 
is functionally complete for the reader (see Exercises 15 and 16). 
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Exercises 


. Find a Boolean product of the Boolean variables x, y, 


and z, or their complements, that has the value 1 if and 
only if 

b) x =0,y=1,z=0. 
d) x=y=z=0. 


a) x=y=0,z=1. 
ec) x=0,y=z=1. 


. Find the sum-of-products expansions of these Boolean 
functions. 
a) F(x,y)=xX+y b) FQ, y) =xy 
c) F(x, y)=1 d) F(x, y)=y 


. Find the sum-of-products expansions of these Boolean 


functions. 


a) F(x, y,z)=x+y+z 
b) FQ, y,z)=(&+2)y 
ce) F(x, y,z)=x 

d) F(x, y,2=xy 


. Find the sum-of-products expansions of the Boolean 


function F(x, y, z) that equals 1 if and only if 


a) x=0. b) xy = 0. 
ec) x+y=0. d) xyz =0. 


. Find the sum-of-products expansion of the Boolean func- 


tion F(w, x, y, z) that has the value 1 if and only if an 
odd number of w, x, y, and z have the value 1. 


. Find the sum-of-products expansion of the Boolean func- 


tion F(x1, x2, x3, x4, x5) that has the value 1| if and only 
if three or more of the variables x1, x2, x3, x4, and x5 have 
the value 1. 


Another way to find a Boolean expression that represents a 
Boolean function is to form a Boolean product of Boolean 
sums of literals. Exercises 7-11 are concerned with represen- 
tations of this kind. 


7. 


Find a Boolean sum containing either x or X, either y 
or y, and either z or Z that has the value 0 if and only if 
a) x=y=1,z=0. b) x=y=z=0. 
ec) x=z=0,y=1. 


. Find a Boolean product of Boolean sums of literals that 


has the value 0 if and only if x = y=1 and z=0, 
x =z=0Oandy=1,orx = y =z =0. [Hint: Take the 
Boolean product of the Boolean sums found in parts (a), 
(b), and (c) in Exercise 7.] 


Logic Gates 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


. Show that the Boolean sum yj + yo +---+ y,, where 


yj = xX; Or yj = X;, has the value 0 for exactly one combi- 
nation of the values of the variables, namely, when x; = 0 
if y, =x; and x; = 1 if y; = x;. This Boolean sum is 
called a maxterm. 


Show that a Boolean function can be represented as 
a Boolean product of maxterms. This representation is 
called the product-of-sums expansion or conjunctive 
normal form of the function. [Hint: Include one max- 
term in this product for each combination of the variables 
where the function has the value 0.] 


Find the product-of-sums expansion of each of the 
Boolean functions in Exercise 3. 


Express each of these Boolean functions using the oper- 
ators - and ~. 


a) x+y+z b) x + y@+4+2z) 
x+y d) x(x +y+2Z) 
Express each of the Boolean functions in Exercise 12 us- 


ing the operators + and ~. 
Show that 

a) x=x|x. 

ce) xty=Ol/x)| (ly). 
Show that 

a) X=xJ)x. 

b) xy=(r Lx) 1 1). 
eo xty=Aly)louly). 

Show that { | } is functionally complete using Exer- 
cise 15. 


b) xy=(x|y)| @Iy). 


Express each of the Boolean functions in Exercise 3 using 
the operator | . 


Express each of the Boolean functions in Exercise 3 using 
the operator |. 

Show that the set of operators {+, -} is not functionally 
complete. 


Are these sets of operators functionally complete? 
a) {+, @} b) C, @} c) {-, @} 


Introduction 


Links ~@ 


Boolean algebra is used to model the circuitry of electronic devices. Each input and each output 
of such a device can be thought of as a member of the set {0, 1}. A computer, or other electronic 


device, is made up of a number of circuits. Each circuit can be designed using the rules of 
Boolean algebra that were studied in Sections 12.1 and 12.2. The basic elements of circuits 


EXAMPLE 1 
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(a) Inverter (b) OR gate (c) AND gate 


FIGURE 1 Basic Types of Gates. 


are called gates, and were introduced in Section 1.2. Each type of gate implements a Boolean 
operation. In this section we define several types of gates. Using these gates, we will apply the 
rules of Boolean algebra to design circuits that perform a variety of tasks. The circuits that we 
will study in this chapter give output that depends only on the input, and not on the current 
state of the circuit. In other words, these circuits have no memory capabilities. Such circuits are 
called combinational circuits or gating networks. 

We will construct combinational circuits using three types of elements. The first is an 
inverter, which accepts the value of one Boolean variable as input and produces the complement 
of this value as its output. The symbol used for an inverter is shown in Figure 1(a). The input to 
the inverter is shown on the left side entering the element, and the output is shown on the right 
side leaving the element. 

The next type of element we will use is the OR gate. The inputs to this gate are the values 
of two or more Boolean variables. The output is the Boolean sum of their values. The symbol 
used for an OR gate is shown in Figure 1(b). The inputs to the OR gate are shown on the left 
side entering the element, and the output is shown on the right side leaving the element. 

The third type of element we will use is the AND gate. The inputs to this gate are the values 
of two or more Boolean variables. The output is the Boolean product of their values. The symbol 
used for an AND gate is shown in Figure 1(c). The inputs to the AND gate are shown on the left 
side entering the element, and the output is shown on the right side leaving the element. 

We will permit multiple inputs to AND and OR gates. The inputs to each of these gates are 
shown on the left side entering the element, and the output is shown on the right side. Examples 
of AND and OR gates with n inputs are shown in Figure 2. 


— x 
wa > XXQ099X, s— Xp +X teeetx, 
oo Xn) ——>. 


FIGURE 2 Gates with n Inputs. 


Combinations of Gates 


Combinational circuits can be constructed using a combination of inverters, OR gates, and AND 
gates. When combinations of circuits are formed, some gates may share inputs. This is shown in 
one of two ways in depictions of circuits. One method is to use branchings that indicate all the 
gates that use a given input. The other method is to indicate this input separately for each gate. 
Figure 3 illustrates the two ways of showing gates with the same input values. Note also that 
output from a gate may be used as input by one or more other elements, as shown in Figure 3. 
Both drawings in Figure 3 depict the circuit that produces the output xy + xy. 


Construct circuits that produce the following outputs: (a) (x + y)x, (b) ¥ (y + Z), and (c) (x + 
ytz)(x yz). 


Solution: Circuits that produce these outputs are shown in Figure 4. < 
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FIGURE 4 _ Circuits that Produce the Outputs Specified in Example 1. 
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Examples of Circuits 


We will give some examples of circuits that perform some useful functions. 


A committee of three individuals decides issues for an organization. Each individual votes either 
yes or no for each proposal that arises. A proposal is passed if it receives at least two yes votes. 
Design a circuit that determines whether a proposal passes. 


Solution: Let x = 1 if the first individual votes yes, and x = 0 if this individual votes no; 
let y = 1 if the second individual votes yes, and y = 0 if this individual votes no; let z = 1 
if the third individual votes yes, and z = 0 if this individual votes no. Then a circuit must be 
designed that produces the output | from the inputs x, y, and z when two or more of x, y, 
and z are 1. One representation of the Boolean function that has these output values is 
xy +xz-+ yz (see Exercise 12 in Section 12.1). The circuit that implements this function is 
shown in Figure 5. < 


pS 
> Pp xytxzct+yz 


FIGURE 5 A Circuit for Majority Voting. 


Sometimes light fixtures are controlled by more than one switch. Circuits need to be designed 
so that flipping any one of the switches for the fixture turns the light on when it is off and turns 
the light off when it is on. Design circuits that accomplish this when there are two switches and 
when there are three switches. 


Solution: We will begin by designing the circuit that controls the light fixture when two different 
switches are used. Let x = 1 when the first switch is closed and x = 0 when it is open, and let 
y = | when the second switch is closed and y = 0 when it is open. Let F(x, y) = 1 when the 
light is on and F(x, y) = 0 when it is off. We can arbitrarily decide that the light will be on 
when both switches are closed, so that F(1, 1) = 1. This determines all the other values of F. 
When one of the two switches is opened, the light goes off, so F(1, 0) = F(O, 1) = 0. When 
the other switch is also opened, the light goes on, so F'(0, 0) = 1. Table 1 displays these values. 
Note that F(x, y) = xy + x y. This function is implemented by the circuit shown in Figure 6. 


x — >> 
y ——_»|_/ 


x p>) >o : 
y p)>o 


FIGURE 6 A Circuit for a Light Controlled by Two Switches. 
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FIGURE 7 A Circuit for a Fixture Controlled by Three Switches. 


We will now design a circuit for three switches. Let x, y, and z be the Boolean variables that 
indicate whether each of the three switches is closed. We let x = 1 when the first switch is closed, 
and x = 0 when it is open; y = | when the second switch is closed, and y = 0 when it is open; 
and z = | when the third switch is closed, and z = 0 when itis open. Let F(x, y, z) = 1 when the 
light is on and F(x, y, z) = 0 when the light is off. We can arbitrarily specify that the light be on 
when all three switches are closed, so that F(1, 1, 1) = 1. This determines all other values of F’. 
When one switch is opened, the light goes off, so F(1, 1,0) = Fd, 0, 1) = FO, 1, 1) =0. 
When a second switch is opened, the light goes on, so F(1, 0, 0) = F(0, 1,0) = F(0, 0, 1) = 1. 
Finally, when the third switch is opened, the light goes off again, so F(0, 0, 0) = 0. Table 2 
shows the values of this function. 

The function F can be represented by its sum-of-products expansion as F(x, y, z) = 
xyz+xyzZ+xyz+Xx yz. The circuit shown in Figure 7 implements this function. 


Adders 


We will illustrate how logic circuits can be used to carry out addition of two positive integers 
from their binary expansions. We will build up the circuitry to do this addition from some 
component circuits. First, we will build a circuit that can be used to find x + y, where x and y 
are two bits. The input to our circuit will be x and y, because these each have the value 0 or the 
value |. The output will consist of two bits, namely, s and c, where s is the sum bit and c is the 
carry bit. This circuit is called a multiple output circuit because it has more than one output. 
The circuit that we are designing is called the half adder, because it adds two bits, without 
considering a carry from a previous addition. We show the input and output for the half adder 
in Table 3. From Table 3 we see that c = xy and that s = xy + Xy = (x + y)(xy). Hence, the 
circuit shown in Figure 8 computes the sum bit s and the carry bit c from the bits x and y. 

We use the full adder to compute the sum bit and the carry bit when two bits and a carry 
are added. The inputs to the full adder are the bits x and y and the carry c;. The outputs are the 
sum bit s and the new carry cj+1. The inputs and outputs for the full adder are shown in Table 4. 
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FIGURE 8 The Half Adder. FIGURE 9 A Full Adder. 
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the Full Adder. 
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Exercises 


The two outputs of the full adder, the sum bit s and the carry cj+ 1, are given by the sum- 
of-products expansions xyc; + xy Cj; +Xyc; +X yc; and xycj + xyc; + xyc; + Xyc;, respec- 
tively. However, instead of designing the full adder from scratch, we will use half adders to 
produce the desired output. A full adder circuit using half adders is shown in Figure 9. 

Finally, in Figure 10 we show how full and half adders can be used to add the two three-bit 
integers (x2x1xq)2 and (v2 yj yo)2 to produce the sum (53525)59)2. Note that s3, the highest-order 
bit in the sum, is given by the carry c2. 


oP) Bale |. Po 

Yo. —p| adder }—p| > 5; 
Full 

xy > 
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y1 p> : >> 
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FIGURE 10 Adding Two Three-Bit 
Integers with Full and Half Adders. 
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6. Construct circuits from inverters, AND gates, and OR 
gates to produce these outputs. 


a) x+y b) X+ y)x 
Cc) xyz +X YZ d) &@+2)(y+2Z) 


7. Design a circuit that implements majority voting for five 
individuals. 


8. Design a circuit for a light fixture controlled by four 
switches, where flipping one of the switches turns the 
light on when it is off and turns it off when it is on. 

9. Show how the sum of two five-bit integers can be found 
using full and half adders. 


10. Construct a circuit for a half subtractor using AND gates, 
OR gates, and inverters. A half subtractor has two bits 
as input and produces as output a difference bit and a 
borrow. 


11. Construct a circuit for a full subtractor using AND gates, 
OR gates, and inverters. A full subtractor has two bits 
and a borrow as input, and produces as output a difference 
bit and a borrow. 


12. Use the circuits from Exercises 10 and 11 to find the dif- 
ference of two four-bit integers, where the first integer is 
greater than the second integer. 


* 13. Construct a circuit that compares the two-bit integers 
(x1x9)2 and (yj; yo)2, returning an output of 1 when the 
first of these numbers is larger and an output of 0 other- 
wise. 

*14, Construct a circuit that computes the product of the two- 
bit integers (x,x9)2 and (1 yo)2. The circuit should have 
four output bits for the bits in the product. 


Minimization of Circuits 


Two gates that are often used in circuits are NAND and NOR 
gates. When NAND or NOR gates are used to represent cir- 
cuits, no other types of gates are needed. The notation for these 
gates is as follows: 


i x NAND y F x NOR y 
¥ —— FP y —»> 


*15. Use NAND gates to construct circuits with these out- 


puts. 
a) x b) x+y 
c) xy d) x@y 


* 16. Use NOR gates to construct circuits for the outputs given 
in Exercise 15. 


*17. Construct a half adder using NAND gates. 


* 18. Construct a half adder using NOR gates. 


A multiplexer is a switching circuit that produces as output 
one of a set of input bits based on the value of control bits. 


19. Construct a multiplexer using AND gates, OR gates, and 
inverters that has as input the four bits xo, x1, x2, and x3 
and the two control bits co and c;. Set up the circuit so 
that x; is the output, where i is the value of the two-bit 
integer (c1Co)2. 


The depth of a combinatorial circuit can be defined by spec- 
ifying that the depth of the initial input is 0 and if a gate 
has n different inputs at depths d), do,..., dy, respectively, 
then its outputs have depth equal to max(d,, do,..., dn) + 1; 
this value is also defined to be the depth of the gate. The depth 
of a combinatorial circuit is the maximum depth of the gates 
in the circuit. 


20. Find the depth of 


a) the circuit constructed in Example 2 for majority vot- 
ing among three people. 


b) the circuit constructed in Example 3 for a light con- 
trolled by two switches. 


c) the half adder shown in Figure 8. 
d) the full adder shown in Figure 9. 


Introduction 


The efficiency of a combinational circuit depends on the number and arrangement of its gates. The 
process of designing a combinational circuit begins with the table specifying the output for each 
combination of input values. We can always use the sum-of-products expansion of a circuit to 
find a set of logic gates that will implement this circuit. However, the sum-of-products expansion 
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x : 
XYZ 
<- xyz + xyz 


FIGURE 1 Two Circuits with the Same Output. 


may contain many more terms than are necessary. Terms in a sum-of-products expansion that 
differ in just one variable, so that in one term this variable occurs and in the other term the 
complement of this variable occurs, can be combined. For instance, consider the circuit that has 
output | if and only if x = y = z = l orx =z = 1 and y = 0. The sum-of-products expansion 
of this circuit is xyz + xyz. The two products in this expansion differ in exactly one variable, 
namely, y. They can be combined as 


xyz t+ xyz = (y + y)(xz) 
= 1- (xz) 
= XZ. 


Hence, xz is a Boolean expression with fewer operators that represents the circuit. We show 
two different implementations of this circuit in Figure |. The second circuit uses only one gate, 
whereas the first circuit uses three gates and an inverter. 

This example shows that combining terms in the sum-of-products expansion of a circuit 
leads to a simpler expression for the circuit. We will describe two procedures that simplify 
sum-of-products expansions. 

The goal of both procedures is to produce Boolean sums of Boolean products that represent 
a Boolean function with the fewest products of literals such that these products contain the 
fewest literals possible among all sums of products that represent a Boolean function. Finding 
such a sum of products is called minimization of the Boolean function. Minimizing a Boolean 
function makes it possible to construct a circuit for this function that uses the fewest gates and 
fewest inputs to the AND gates and OR gates in the circuit, among all circuits for the Boolean 
expression we are minimizing. 

Until the early 1960s logic gates were individual components. To reduce costs it was impor- 
tant to use the fewest gates to produce a desired output. However, in the mid-1960s, integrated 
circuit technology was developed that made it possible to combine gates on a single chip. Even 
though it is now possible to build increasingly complex integrated circuits on chips at low cost, 
minimization of Boolean functions remains important. 

Reducing the number of gates on a chip can lead to a more reliable circuit and can reduce 
the cost to produce the chip. Also, minimization makes it possible to fit more circuits on the 
same chip. Furthermore, minimization reduces the number of inputs to gates in a circuit. This 
reduces the time used by a circuit to compute its output. Moreover, the number of inputs to a 
gate may be limited because of the particular technology used to build logic gates. 

The first procedure we will introduce, known as Karnaugh maps (or K-maps), was designed 
in the 1950s to help minimize circuits by hand. K-maps are useful in minimizing circuits with 
up to six variables, although they become rather complex even for five or six variables. The 
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second procedure we will describe, the Quine-McCluskey method, was invented in the 1960s. 
It automates the process of minimizing combinatorial circuits and can be implemented as a 
computer program. 


COMPLEXITY OF BOOLEAN FUNCTION MINIMIZATION Unfortunately, minimizing 
Boolean functions with many variables is a computationally intensive problem. It has been shown 
that this problem is an NP-complete problem (see Section 3.3 and [Ka93]), so the existence of a 
polynomial-time algorithm for minimizing Boolean circuits is unlikely. The Quine—McCluskey 
method has exponential complexity. In practice, it can be used only when the number of liter- 
als does not exceed ten. Since the 1970s a number of newer algorithms have been developed 
for minimizing combinatorial circuits (see [Ha93] and [KaBe04]). However, with the best al- 
gorithms yet devised, only circuits with no more than 25 variables can be minimized. Also, 
heuristic (or rule-of-thumb) methods can be used to substantially simplify, but not necessarily 
minimize, Boolean expressions with a larger number of literals. 


Karnaugh Maps 


To reduce the number of terms in a Boolean expression representing a circuit, it is necessary 
to find terms to combine. There is a graphical method, called a Karnaugh map or K-map, 
for finding terms to combine for Boolean functions involving a relatively small number of 
variables. The method we will describe was introduced by Maurice Karnaugh in 1953. His 
method is based on earlier work by E. W. Veitch. (This method is usually applied only when 
the function involves six or fewer variables.) K-maps give us a visual method for simplifying 
sum-of-products expansions; they are not suited for mechanizing this process. We will first 
illustrate how K-maps are used to simplify expansions of Boolean functions in two variables. 
We will continue by showing how K-maps can be used to minimize Boolean functions in three 
variables and then in four variables. Then we will describe the concepts that can be used to 
extend K-maps to minimize Boolean functions in more than four variables. 

There are four possible minterms in the sum-of-products expansion of a Boolean function 
in the two variables x and y. A K-map for a Boolean function in these two variables consists 
of four cells, where a | is placed in the cell representing a minterm if this minterm is present 
in the expansion. Cells are said to be adjacent if the minterms that they represent differ in 
exactly one literal. For instance, the cell representing xy is adjacent to the cells representing xy 
and x y. The four cells and the terms that they represent are shown in Figure 2. 


Find the K-maps for (a) xy + xy, (b) xy + xy, and (c)xy+xy+xXy. 


Solution: We include a | in a cell when the minterm represented by this cell is present in the 
sum-of-products expansion. The three K-maps are shown in Figure 3. < 


We can identify minterms that can be combined from the K-map. Whenever there are Is 
in two adjacent cells in the K-map, the minterms represented by these cells can be combined 
into a product involving just one of the variables. For instance, xy and xX y are represented by 
adjacent cells and can be combined into y, because xy + xX y = (x + X)y = y. Moreover, if Is 


MAURICE KARNAUGH (BORN 1924) Maurice Karnaugh, born in New York City, received his B.S. from 
the City College of New York and his M.S. and Ph.D. from Yale University. He was a member of the technical 
staff at Bell Laboratories from 1952 until 1966 and Manager of Research and Development at the Federal 
Systems Division of AT&T from 1966 to 1970. In 1970 he joined IBM as a member of the research staff. 
Karnaugh has made fundamental contributions to the application of digital techniques in both computing and 
telecommunications. His current interests include knowledge-based systems in computers and heuristic search 
methods. 


EXAMPLE 2 
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(a) (b) (c) 
FIGURE 3 K-maps for the Sum-of-Products Expansions in Example 1. 
are in all four cells, the four minterms can be combined into one term, namely, the Boolean 
expression | that involves none of the variables. We circle blocks of cells in the K-map that 
represent minterms that can be combined and then find the corresponding sum of products. The 


goal is to identify the largest possible blocks, and to cover all the 1s with the fewest blocks using 
the largest blocks first and always using the largest possible blocks. 


Simplify the sum-of-products expansions given in Example 1. 


Solution: The grouping of minterms is shown in Figure 4 using the K-maps for these expansions. 
Minimal expansions for these sums-of-products are (a) y, (b) xy + xy, and (c)x + y. < 
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FIGURE 4 _ Simplifying the Sum-of-Products Expansions from Example 2. 


A K-map in three variables is a rectangle divided into eight cells. The cells represent the 
eight possible minterms in three variables. Two cells are said to be adjacent if the minterms that 
they represent differ in exactly one literal. One of the ways to form a K-map in three variables 
is shown in Figure 5(a). This K-map can be thought of as lying on a cylinder, as shown in 
Figure 5(b). On the cylinder, two cells have a common border if and only if they are adjacent. 


yZ yz yz yz 
% XyZ XYZ XYZ xyz 
Bs XYZ XYZ XYZ XYZ 


(a) 


(b) 


FIGURE 5 K-maps in Three Variables. 
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FIGURE 6 Blocks in K-maps in Three Variables. 


EXAMPLE 3 


To simplify a sum-of-products expansion in three variables, we use the K-map to iden- 
tify blocks of minterms that can be combined. Blocks of two adjacent cells represent pairs of 
minterms that can be combined into a product of two literals; 2 x 2 and 4 x 1| blocks of cells 
represent minterms that can be combined into a single literal; and the block of all eight cells 
represents a product of no literals, namely, the function 1. In Figure 6, | x 2,2 x 1,2 x 2,4x1, 
and 4 x 2 blocks and the products they represent are shown. 

The product of literals corresponding to a block of all 1s in the K-map is called an implicant 
of the function being minimized. It is called a prime implicant if this block of 1s is not contained 
in a larger block of 1s representing the product of fewer literals than in this product. 

The goal is to identify the largest possible blocks in the map and cover all the 1s in the map 
with the least number of blocks, using the largest blocks first. The largest possible blocks are 
always chosen, but we must always choose a block if it is the only block of Is covering a 1 in 
the K-map. Such a block represents an essential prime implicant. By covering all the Is in the 
map with blocks corresponding to prime implicants we can express the sum of products as a 
sum of prime implicants. Note that there may be more than one way to cover all the Is using 
the least number of blocks. 

Example 3 illustrates how K-maps in three variables are used. 


Use K-maps to minimize these sum-of-products expansions. 


(a) xyZ+xXVYZ+XVZ+XYZ 

(b) xyz +xyz+xyz+xyz+xyz 

(Cc) XYZ +XYZ+XYZAXVZAXVZAXYVZ+XYVZ 
(d) xyZ+xYZ+Xyz+xXyzZ 


Solution: The K-maps for these sum-of-products expansions are shown in Figure 7. The group- 
ing of blocks shows that minimal expansions into Boolean sums of Boolean products are 
(a)xz+yz+Xxyz, (b) y+ Xz, (c)x + y +z, and (d) xz +X ¥. In part (d) note that the prime 
implicants xz and xy are essential prime implicants, but the prime implicant yz is a 
prime implicant that is not essential, because the cells it covers are covered by the other two 
prime implicants. < 
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FIGURE 7 Using K-maps in Three Variables. 


A K-map in four variables is a square that is divided into 16 cells. The cells represent the 16 
possible minterms in four variables. One of the ways to form a K-map in four variables is shown 
in Figure 8. 

Two cells are adjacent if and only if the minterms they represent differ in one literal. Con- 
sequently, each cell is adjacent to four other cells. The K-map of a sum-of-products expansion 
in four variables can be thought of as lying on a torus, so that adjacent cells have a common 
boundary (see Exercise 28). The simplification of a sum-of-products expansion in four variables 
is carried out by identifying those blocks of 2, 4, 8, or 16 cells that represent minterms that can be 
combined. Each cell representing a minterm must either be used to form a product using fewer 
literals, or be included in the expansion. In Figure 9 some examples of blocks that represent 
products of three literals, products of two literals, and a single literal are illustrated. 

As is the case in K-maps in two and three variables, the goal is to identify the largest 
blocks of 1s in the map that correspond to the prime implicants and to cover all the 1s using 
the fewest blocks needed, using the largest blocks first. The largest possible blocks are always 
used. Example 4 illustrates how K-maps in four variables are used. 


wx | wxyz | wxyzZ | wxyz | wxyz 


WX | Wxyz | WXYZ | WxyzZ | Wxyz 


Wx| wxyz | WxyZ | wxyzZ | wxyz 


FIGURE 8 K-maps in Four Variables. 
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EXAMPLE 4 


yz yz yz yz yz yz yz YZ 
wx wx 
wx wx 
wx wx 
Wx wx 
WXZ = WXYZ + WxyZ Wx = wxyz + WxyZ + 
WKYZ + WXYZ 
(a) (b) 
yz yz yz yz yz yz yz yz 
wx wx 
wx wx 
wx wx 
wx wx 
XZ = wxyz + wxyz + Z = WXYZ + WXYZ + WXYZ + 
Wxyz + WxyZz WXYZ + WXYZ + WXYZ + WXYZ + WXYZ 


(c) (d) 


FIGURE 9 Blocks in K-maps in Four Variables. 


Use K-maps to simplify these sum-of-products expansions. 


(a) wxyz + wxyzZ+wxyzZt+wxyzt+wxyztwxyzt+wxyzt 
Wxyz+Wwxyz 
(b) wxyz+wxyz+wxyztwxyzt+wxyzt+wxyzt+wxyz 


(c) wxyZ +wxyZ+wxyz+wxyzt+wxyzZ+wxyz+wrxyzt+wxyzZt 
WXYZ +WXHXYZ+WHXYZ 


Solution: The K-maps for these expansions are shown in Figure 10. Using the blocks shown 
leads to the sum of products (a) wyz + wxz + wx y+wxy + wxyz, (b) yz +wxy + XZ, and 
(c) z+ wx + wxy. The reader should determine whether there are other choices of blocks in 
each part that lead to different sums of products representing these Boolean functions. < 


K-maps can realistically be used to minimize Boolean functions with five or six variables, 
but beyond that, they are rarely used because they become extremely complicated. However, the 
concepts used in K-maps play an important role in newer algorithms. Furthermore, mastering 
these concepts helps you understand these newer algorithms and the computer-aided design 
(CAD) programs that implement them. As we develop these concepts, we will be able to illustrate 
them by referring back to our discussion of minimization of Boolean functions in three and in 
four variables. 

The K-maps we used to minimize Boolean functions in two, three, and four variables are 
built using 2 x 2,2 x 4,and4 x 4 rectangles, respectively. Furthermore, corresponding cells in 
the top row and bottom row and in the leftmost column and rightmost column in each of these 


EXAMPLE 5 


EXAMPLE 6 
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FIGURE 10 Using K-maps in Four Variables. 


cases are considered adjacent because they represent minterms differing in only one literal. We 
can build K-maps for minimizing Boolean functions in more than four variables in a similar 
way. We use a rectangle containing 2!"/2! rows and 2!"/21 columns. (These K-maps contain 
2” cells because [n/2] + |[n/2] =n.) The rows and columns need to be positioned so that the 
cells representing minterms differing in just one literal are adjacent or are considered adjacent 
by specifying additional adjacencies of rows and columns. To help (but not entirely) achieve 
this, the rows and columns of a K-map are arranged using Gray codes (see Section 10.5), where 
we associate bit strings and products by specifying that a 1 corresponds to the appearance of 
a variable and a 0 with the appearance of its complement. For example, in a 10-dimensional 
K-map, the Gray code 01110 used to label a row corresponds to the product ¥1.x2x3x4X5. 


The K-maps we used to minimize Boolean functions with four variables have four rows and 
four columns. Both the rows and the columns are arranged using the Gray code 11,10,00,01. 
The rows represent products wx, wx, wx, and wx, respectively, and the columns correspond to 
the products yz, yz, yz, and yz, respectively. Using Gray codes and considering cells adjacent 
in the first and last rows and in the first and last columns, we ensured that minterms that differ 


in only one variable are always adjacent. 


To minimize Boolean functions in five variables we use K-maps with 2? = 8 columns and 
2? = 4 rows. We label the four rows using the Gray code 11,10,00,01, corresponding to 
X1X2, X1X2, X1X2, and X1x2, respectively. We label the eight columns using the Gray code 
111,110,100,101,001,000,010,011 corresponding to the terms x3%4.x5, X3X4%5, X3X4X5, X3X4X5, 
X3X4X5, X3X4X5, X3XAX5, and X3X4Xx5, respectively. Using Gray codes to label columns and rows 
ensures that the minterms represented by adjacent cells differ in only one variable. However, 
to make sure all cells representing products that differ in only one variable are considered ad- 
jacent, we consider cells in the top and bottom rows to be adjacent, as well as cells in the first 
and eighth columns, the first and fourth columns, the second and seventh columns, the third and 
sixth columns, and the fifth and eighth columns (as the reader should verify). < 


To use a K-map to minimize a Boolean function in n variables, we first draw a K-map of 
the appropriate size. We place Is in all cells corresponding to minterms in the sum-of-products 
expansion of this function. We then identify all prime implicants of the Boolean function. To do 
this we look for the blocks consisting of 2* clustered cells all containing a1, where 1 < k <n. 
These blocks correspond to the product of n — k literals. (Exercise 33 asks the reader to verify 
this.) Furthermore, a block of 2* cells each containing a 1 not contained in a block of 2*+! 
cells each containing a | represents a prime implicant. The reason that this implicant is a prime 
implicant is that no product obtained by deleting a literal is also represented by a block of cells 
all containing Is. 
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EXAMPLE 7 


Links 


EXAMPLE 8 


A block of eight cells representing a product of two literals in a K-map for minimizing Boolean 
functions in five variables all containing Is is a prime implicant if it is not contained in a larger 
block of 16 cells all containing Is representing a single literal. 


Once all prime implicants have been identified, the goal is to find the smallest possible subset 
of these prime implicants with the property that the cells representing these prime implicants 
cover all the cells containing a | in the K-map. We begin by selecting the essential prime 
implicants because each of these is represented by a block that covers a cell containing a | that 
is not covered by any other prime implicant. We add additional prime implicants to ensure that 
all 1s in the K-map are covered. When the number of variables is large, this last step can become 
exceedingly complicated. 


Don’t Care Conditions 


In some circuits we care only about the output for some combinations of input values, be- 
cause other combinations of input values are not possible or never occur. This gives us freedom 
in producing a simple circuit with the desired output because the output values for all those 
combinations that never occur can be arbitrarily chosen. The values of the function for these 
combinations are called don’t care conditions. A d is used in a K-map to mark those com- 
binations of values of the variables for which the function can be arbitrarily assigned. In the 
minimization process we can assign Is as values to those combinations of the input values that 
lead to the largest blocks in the K-map. This is illustrated in Example 8. 


One way to code decimal expansions using bits is to use the four bits of the binary expansion 
of each digit in the decimal expansion. For instance, 873 is encoded as 100001110011. This 
encoding of a decimal expansion is called a binary coded decimal expansion. Because there 
are 16 blocks of four bits and only 10 decimal digits, there are six combinations of four bits that 
are not used to encode digits. Suppose that a circuit is to be built that produces an output of 1 if 
the decimal digit is 5 or greater and an output of 0 if the decimal digit is less than 5. How can 
this circuit be simply built using OR gates, AND gates, and inverters? 


Solution: Let F(w, x, y, z) denote the output of the circuit, where wxyz is a binary expansion 
of a decimal digit. The values of F are shown in Table |. The K-map for F, with ds in the 
don’t care positions, is shown in Figure 1 1(a). We can either include or exclude squares with 
ds from blocks. This gives us many possible choices for the blocks. For example, excluding 
all squares with ds and forming blocks, as shown in Figure 11(b), produces the expression 
wx y + wxy + wxz. Including some of the ds and excluding others and forming blocks, as 
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FIGURE 11 The K-map for F Showing Its Don’t Care Positions. 


shown in Figure 11(c), produces the expression wx + wxy + xyz. Finally, including all the ds 
and using the blocks shown in Figure 11(d) produces the simplest sum-of-products expansion 
possible, namely, F(x, y,z) =w+xy+ xz. < 


The Quine—McCluskey Method 


We have seen that K-maps can be used to produce minimal expansions of Boolean functions 
as Boolean sums of Boolean products. However, K-maps are awkward to use when there are 
more than four variables. Furthermore, the use of K-maps relies on visual inspection to identify 
terms to group. For these reasons there is a need for a procedure for simplifying sum-of-products 
expansions that can be mechanized. The Quine—McCluskey method is such a procedure. It can 
be used for Boolean functions in any number of variables. It was developed in the 1950s by 
W. V. Quine and E. J. McCluskey, Jr. Basically, the Quine-McCluskey method consists of two 


EDWARD J. McCLUSKEY (BORN 1929) Edward McCluskey attended Bowdoin College and M.LT., where 
he received his doctorate in electrical engineering in 1956. He joined Bell Telephone Laboratories in 1955, 
remaining there until 1959. McCluskey was professor of electrical engineering at Princeton University from 
1959 until 1966, also serving as director of the Computer Center at Princeton from 1961 to 1966. In 1967 he 
took a position as professor of computer science and electrical engineering at Stanford University, where he also 
served as director of the Digital Systems Laboratory from 1969 to 1978. McCluskey has worked in a variety of 
areas in computer science, including fault-tolerant computing, computer architecture, testing, and logic design. 
He is the director of the Center for Reliable Computing at Stanford University where he is now an emeritus 
professor. McCluskey is also an ACM Fellow. 
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TABLE 2 
Minterm Number of Is 


parts. The first part finds those terms that are candidates for inclusion in a minimal expansion 
as a Boolean sum of Boolean products. The second part determines which of these terms to 
actually use. We will use Example 9 to illustrate how, by successively combining implicants 
into implicants with one fewer literal, this procedure works. 


EXAMPLE9 We will show how the Quine-McCluskey method can be used to find a minimal expansion 
equivalent to 


XYZ+XVZAXYVZ +X YZ +X YZ. 


We will represent the minterms in this expansion by bit strings. The first bit will be 1 if x 
occurs and 0 if x occurs. The second bit will be | if y occurs and 0 if y occurs. The third bit 
will be 1 if z occurs and 0 if Z occurs. We then group these terms according to the number of Is 
in the corresponding bit strings. This information is shown in Table 2. 

Minterms that can be combined are those that differ in exactly one literal. Hence, two terms 
that can be combined differ by exactly one in the number of |s in the bit strings that represent 
them. When two minterms are combined into a product, this product contains two literals. A 
product in two literals is represented using a dash to denote the variable that does not occur. For 
instance, the minterms xyz and X yz, represented by bit strings 101 and 001, can be combined 
into yz, represented by the string —O1. All pairs of minterms that can be combined and the 
product formed from these combinations are shown in Table 3. 

Next, all pairs of products of two literals that can be combined are combined into one 
literal. Two such products can be combined if they contain literals for the same two variables, 
and literals for only one of the two variables differ. In terms of the strings representing the 
products, these strings must have a dash in the same position and must differ in exactly one 
of the other two slots. We can combine the products yz and yz, represented by the strings —-11 
and —01, into z, represented by the string —- —1. We show all the combinations of terms that can 
be formed in this way in Table 3. 


TABLE 3 


(1,2,3,4) 
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TABLE 4 


In Table 3 we also indicate which terms have been used to form products with fewer literals; 
these terms will not be needed in a minimal expansion. The next step is to identify a minimal 
set of products needed to represent the Boolean function. We begin with all those products that 
were not used to construct products with fewer literals. Next, we form Table 4, which has a row 
for each candidate product formed by combining original terms, and a column for each original 
term; and we put an X in a position if the original term in the sum-of-products expansion was 
used to form this candidate product. In this case, we say that the candidate product covers 
the original minterm. We need to include at least one product that covers each of the original 
minterms. Consequently, whenever there is only one X in a column in the table, the product 
corresponding to the row this X is in must be used. From Table 4 we see that both z and x y are 
needed. Hence, the final answer is z+ X y. < 


As was illustrated in Example 9, the Quine—McCluskey method uses this sequence of steps 
to simplify a sum-of-products expression. 


1. Express each minterm in n variables by a bit string of length n with a | in the ith position 
if x; occurs and a 0 in this position if x; occurs. 


2. Group the bit strings according to the number of Is in them. 


3. Determine all products in n — 1 variables that can be formed by taking the Boolean sum 
of minterms in the expansion. Minterms that can be combined are represented by bit 
strings that differ in exactly one position. Represent these products in n — 1 variables 
with strings that have a | in the ith position if x; occurs in the product, a 0 in this position 
if x; occurs, and a dash in this position if there is no literal involving x; in the product. 


WILLARD VAN ORMAN QUINE (1908-2000) Willard Quine, born in Akron, Ohio, attended Oberlin College 
and later Harvard University, where he received his Ph.D. in philosophy in 1932. He became a Junior Fellow 
at Harvard in 1933 and was appointed to a position on the faculty there in 1936. He remained at Harvard 
his entire professional life, except for World War II, when he worked for the U.S. Navy decrypting messages 
from German submarines. Quine was always interested in algorithms, but not in hardware. He arrived at his 
discovery of what is now called the Quine—McCluskey method as a device for teaching mathematical logic, 
rather than as a method for simplifying switching circuits. Quine was one of the most famous philosophers of 
the twentieth century. He made fundamental contributions to the theory of knowledge, mathematical logic and 
set theory, and the philosophies of logic and language. His books, including New Foundations of Mathematical 
Logic published in 1937 and Word and Object published in 1960, have had a profound impact. Quine retired from Harvard in 1978 
but continued to commute from his home in Beacon Hill to his office there. He used the 1927 Remington typewriter on which he 
prepared his doctoral thesis for his entire life. He even had an operation performed on this machine to add a few special symbols, 
removing the second period, the second comma, and the question mark. When asked whether he missed the question mark, he 
replied, “Well, you see, I deal in certainties.” There is even a word quine, defined in the New Hacker’s Dictionary as a program 
that generates a copy of its own source code as its complete output. Producing the shortest possible quine in a given programming 
language is a popular puzzle for hackers. 
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EXAMPLE 10 


4. Determine all products in n — 2 variables that can be formed by taking the Boolean sum 
of the products inn — | variables found in the previous step. Products inn — | variables 
that can be combined are represented by bit strings that have a dash in the same position 
and differ in exactly one position. 

5. Continue combining Boolean products into products in fewer variables as long as possible. 


6. Find all the Boolean products that arose that were not used to form a Boolean product in 
one fewer literal. 


7. Find the smallest set of these Boolean products such that the sum of these products 
represents the Boolean function. This is done by forming a table showing which minterms 
are covered by which products. Every minterm must be covered by at least one product. 
The first step in using this table is to find all essential prime implicants. Each essential 
prime implicant must be included because it is the only prime implicant that covers one of 
the minterms. Once we have found essential prime implicants, we can simplify the table 
by eliminating the rows for minterms covered by these prime implicants. Furthermore, 
we can eliminate any prime implicants that cover a subset of minterms covered by another 
prime implicant (as the reader should verify). Moreover, we can eliminate from the table 
the row for a minterm if there is another minterm that is covered by a subset of the 
prime implicants that cover this minterm. This process of identifying essential prime 
implicants that must be included, followed by eliminating redundant prime implicants 
and identifying minterms that can be ignored, is iterated until the table does not change. 
At this point we use a backtracking procedure to find the optimal solution where we add 
prime implicants to the cover to find possible solutions, which we compare to the best 
solution found so far at each step. 


A final example will illustrate how this procedure is used to simplify a sum-of-products 
expansion in four variables. 


Use the Quine—McCluskey method to simplify the sum-of-products expansion wx yz + wxyz + 
WXYZ + WXYZ + WXYZ +WXYZ+WX YZ. 


Solution: We first represent the minterms by bit strings and then group these terms together 
according to the number of Is in the bit strings. This is shown in Table 5. All the Boolean 
products that can be formed by taking Boolean sums of these products are shown in Table 6. 

The only products that were not used to form products in fewer variables are wz, wyz, 
wxy, and xyz. In Table 7 we show the minterms covered by each of these products. To cover 
these minterms we must include wz and wyz, because these products are the only products that 
cover wxyz and wx yz, respectively. Once these two products are included, we see that only 
one of the two products left is needed. Consequently, we can take either wz + wyz + wxy or 
wz+wyz+xyz as the final answer. 


TABLE 5 
Term Bit String 


WXYZ 
WXYZ 


WXYZ 


1110 
1011 
O111 
1010 
0101 
0011 
0001 
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TABLE 6 
a 


(3,5,6,7) 


Exercises 
1. a) Draw a K-map for a function in two variables and put 5. a) Draw a K-map for a function in three variables. Put a 
a | in the cell representing xy. 1 in the cell that represents x yz. 


b) What are the minterms represented by cells adjacent b) Which minterms are represented by cells adjacent to 
to this cell? this cell? 


2. Find the sum-of-products expansions represented by each 
of these K-maps. 6. Use K-maps to find simpler circuits with the same output 


= = as each of the circuits shown. 
a) y y b) y y c) y 


x| 1 x| 1 1 x| 1 1 ) => \ 
x} 1 | x z{ 1 | 1 
x—por—~ x 
<i 


3. Draw the K-maps of these sum-of-products expansions 
in two variables. 


a) xy b) xy +xy 


| 


xyz + Xyz 


c) xy txytxy+xy 


XYZ + xVZ + XyZ + XYZ 


x . 
y—po }— 
4. Use a K-map to find a minimal expansion as a Boolean z—Bo~ L< 
sum of Boolean products of each of these functions of the 5) 
Boolean variables x and y. 
a) xy+xy 


b y ; x 
oe : Rey 


ce) xy +xy+xy+xy 


842 


10. 


11. 


12. 


13. 


14. 
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9 7 Dob yz aye +2) + 5+) 


oy 


(x+Z)+9+2) 


. Draw the K-maps of these sum-of-products expansions 


in three variables. 
a) xyZ 
Cc) xyz+xyzt+xXyZ+xyz 


b) Xyz+XyZ 


. Construct a K-map for F(x, y, z) =xz+ yz+ xyz. Use 


this K-map to find the implicants, prime implicants, and 
essential prime implicants of F(x, y, z). 


. Construct a K-map for F(x, y, z) =xzZ+ xyz + yz. Use 


this K-map to find the implicants, prime implicants, and 
essential prime implicants of F(x, y, z). 


Draw the 3-cube Q3 and label each vertex with the 
minterm in the Boolean variables x, y, and z associated 
with the bit string represented by this vertex. For each 
literal in these variables indicate the 2-cube Q> that is a 
subgraph of Q3 and represents this literal. 


Draw the 4-cube Q4 and label each vertex with the 
minterm in the Boolean variables w, x, y, and z asso- 
ciated with the bit string represented by this vertex. For 
each literal in these variables, indicate which 3-cube Q3 
that is a subgraph of Q4 represents this literal. Indicate 
which 2-cube Q> that is a subgraph of Q4 represents the 
products wz, xy, and yz. 


Use a K-map to find a minimal expansion as a Boolean 
sum of Boolean products of each of these functions in the 
variables x, y, and z. 


a) Xyz+Xyz 
b) xyz + xyz +XxXyz + XxXyz 
Cc) xyZ+xyz+xyzZ+xXyz+xXyz 


d) xyzt+xyzt+xyzZt+xyz+xXyZ+xXyZ 


a) Draw a K-map for a function in four variables. Put a 
1 in the cell that represents Wx yZ. 


b) Which minterms are represented by cells adjacent to 
this cell? 


Use a K-map to find a minimal expansion as a Boolean 
sum of Boolean products of each of these functions in the 
variables w, x, y, and z. 
a) wxyz+wxyz+ wxyZ +wxyz + wx yz 
b) wxyzZ + wxyz + wxyz+ wxyz+WwxyzZ+wxyz 
c) wxyz+wxyzZ+wxyz+wxyz+wxeyzt 
WxYZ+WXyYZ+ WX yz 
d) wxyz+ wxyz + wxyz + wxyz + wxyzt+ 
WXYZ +WXYZ+WXYZ+WX Vz 


15. 


16. 


17. 


18. 


19. 


* 20. 


*21. 


22. 


23. 


24, 


25. 


* 26. 


27. 


* 28. 


Find the cells in a K-map for Boolean functions with five 
variables that correspond to each of these products. 


Cc) X2X4 
d) X3X4 e) x3 f) X5 
How many cells in a K-map for Boolean functions 


with six variables are needed to represent x1, 1X6, 
X1xX2X6, X2X3X4xX5, and xX7xX4X5, respectively? 


a) x1x2x3x4 b) %113x5 


a) How many cells does a K-map in six variables have? 


b) How many cells are adjacent to a given cell ina K-map 
in six variables? 


Show that cells in a K-map for Boolean functions in 
five variables represent minterms that differ in exactly 
one literal if and only if they are adjacent or are in cells 
that become adjacent when the top and bottom rows and 
cells in the first and eighth columns, the first and fourth 
columns, the second and seventh columns, the third and 
sixth columns, and the fifth and eighth columns are con- 
sidered adjacent. 


Which rows and which columns of a 4 x 16 map for 
Boolean functions in six variables using the Gray codes 
1111, 1110, 1010, 1011, 1001, 1000, 0000, 0001, 0011, 
0010, 0110, 0111, 0101, 0100, 1100, 1101 to label the 
columns and 11, 10, 00, 01 to label the rows need to be 
considered adjacent so that cells that represent minterms 
that differ in exactly one literal are considered adjacent? 


Use K-maps to find a minimal expansion as a Boolean 
sum of Boolean products of Boolean functions that have 
as input the binary code for each decimal digit and pro- 
duce as output a | if and only if the digit corresponding 
to the input is 


a) odd. 

c) not 4, 5, or 6. 
Suppose that there are five members on a committee, but 
that Smith and Jones always vote the opposite of Marcus. 


Design a circuit that implements majority voting of the 
committee using this relationship between votes. 


b) not divisible by 3. 


Use the Quine—McCluskey method to simplify the sum- 
of-products expansions in Example 3. 


Use the Quine—McCluskey method to simplify the sum- 
of-products expansions in Exercise 12. 


Use the Quine—McCluskey method to simplify the sum- 
of-products expansions in Example 4. 


Use the Quine—McCluskey method to simplify the sum- 
of-products expansions in Exercise 14. 


Explain how K-maps can be used to simplify product-of- 
sums expansions in three variables. [Hint: Mark with a 0 
all the maxterms in an expansion and combine blocks of 
maxterms. | 


Use the method from Exercise 26 to simplify the product- 
of-sums expansion (x +y+z)(x+y+Zz)(x+y+Z) 
(A+Y+2aty +2). 

Draw a K-map for the 16 minterms in four Boolean vari- 
ables on the surface of a torus. 


29. Build a circuit using OR gates, AND gates, and inverters 
that produces an output of | if a decimal digit, encoded 
using a binary coded decimal expansion, is divisible by 
3, and an output of 0 otherwise. 

In Exercises 30-32 find a minimal sum-of-products expan- 

sion, given the K-map shown with don’t care conditions indi- 

cated with ds. 


30. ye ye yz ¥z 31. ye yZ ye yz 
wx} d|]1}d1 wx] 1 1 
wx d|id wx d| 1 
Wx d|1 Wx l|d 
wx 1 jd wx| d d 


Key Terms and Results 


TERMS 


Boolean variable: a variable that assumes only the values 0 
and 1 


x (complement of x): an expression with the value 1 when x 
has the value 0 and the value 0 when x has the value 1 


x - y (or xy) (Boolean product or conjunction of x and y): 
an expression with the value 1 when both x and y have the 
value | and the value 0 otherwise 


x + y (Boolean sum or disjunction of x and y): an expres- 
sion with the value 1 when either x or y, or both, has the 
value 1, and O otherwise 


Boolean expressions: the expressions obtained recursively by 
specifying that 0, 1, x1, ..., x, are Boolean expressions and 
E}, (E} + E2), and (E; E2) are Boolean expressions if E, 
and E> are 


dual of a Boolean expression: the expression obtained by in- 
terchanging + signs and - signs and interchanging Os and 
Is 


Boolean function of degree n: a function from B” to B where 
B = {0, 1} 

Boolean algebra: a set B with two binary operations V and A, 
elements 0 and 1, and a complementation operator ~ that 
satisfies the identity, complement, associative, commuta- 
tive, and distributive laws 

literal of the Boolean variable x: either x or X 

minterm of x1, X2,...,Xy: a Boolean product yj, y2--- yn, 
where each y; is either x; or X; 

sum-of-products expansion (or disjunctive normal form): 
the representation of a Boolean function as a disjunction of 
minterms 

functionally complete: a set of Boolean operators is called 
functionally complete if every Boolean function can be rep- 
resented using these operators 


x | y (orx NAND y): the expression that has the value 0 when 
both x and y have the value 1 and the value 1 otherwise 
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33. Show that products of k literals correspond to 2”~*- 
dimensional subcubes of the n-cube Q,,, where the ver- 
tices of the cube correspond to the minterms represented 
by the bit strings labeling the vertices, as described in 
Example 8 of Section 10.2. 


x | y (orx NOR y): the expression that has the value 0 when 
either x or y or both have the value | and the value 0 other- 
wise 

inverter: a device that accepts the value of a Boolean variable 
as input and produces the complement of the input 

OR gate: a device that accepts the values of two or more 
Boolean variables as input and produces their Boolean sum 
as output 

AND gate: a device that accepts the values of two or more 
Boolean variables as input and produces their Boolean prod- 
uct as output 

half adder: a circuit that adds two bits, producing a sum bit 
and a carry bit 

full adder: a circuit that adds two bits and a carry, producing 
a sum bit and a carry bit 

K-map for n variables: a rectangle divided into 2” cells where 
each cell represents a minterm in the variables 

minimization of a Boolean function: representing a Boolean 
function as the sum of the fewest products of literals 
such that these products contain the fewest literals possi- 
ble among all sums of products that represent this Boolean 
function 

implicant of a Boolean function: a product of literals with the 
property that if this product has the value 1, then the value 
of this Boolean function is 1 

prime implicant of a Boolean function: a product of literals 
that is an implicant of the Boolean function and no product 
obtained by deleting a literal is also an implicant of this 
function 

essential prime implicant of a Boolean function: a prime 
implicant of the Boolean function that must be included in 
a minimization of this function 

don’t care condition: a combination of input values for a cir- 
cuit that is not possible or never occurs 


RESULTS 


The identities for Boolean algebra (see Table 5 in Section 12.1). 
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An identity between Boolean functions represented by Boolean 
expressions remains valid when the duals of both sides of 
the identity are taken. 

Every Boolean function can be represented by a sum-of- 
products expansion. 


Review Questions 


1. Define a Boolean function of degree n. 
2. How many Boolean functions of degree two are there? 


3. Give a recursive definition of the set of Boolean expres- 
sions. 
4. a) What is the dual of a Boolean expression? 
b) What is the duality principle? How can it be used to 
find new identities involving Boolean expressions? 


5. Explain how to construct the sum-of-products expansion 
of a Boolean function. 


6. a) What does it mean for a set of operators to be func- 
tionally complete? 
b) Is the set {+, -} functionally complete? 
c) Are there sets of a single operator that are functionally 
complete? 


7. Explain how to build a circuit for a light controlled by 
two switches using OR gates, AND gates, and inverters. 


8. Construct a half adder using OR gates, AND gates, and 
inverters. 


Supplementary Exercises 


1. For which values of the Boolean variables x, y, and z does 
a) xt+y+z=xyz? 
b) x(y+z) =x+ yz? 
c) Xyz=x+y+z? 
2. Let x and y belong to {0, 1}. Does it necessarily follow 
that x = y if there exists a value z in {0, 1} such that 
a) xz = yz? b) x+z=y+2? 
©) x@z=yO@z? d)x)z=yJ2z? 
e) x|z=y|z? 


A Boolean function F is called self-dual if and only if 


F(x1,...,Xn) = F(X1,...,%n). 
3. Which of these functions are self-dual? 
a) F(x, y)=x b) F(x, y)=xy+xy 


c) F(x,y)=xt+y d) F(x, y) =xy+xy 
4. Give an example of a self-dual Boolean function of three 
variables. 
*5, How many Boolean functions of degree n are self-dual? 
We define the relation < on the set of Boolean functions 
of degree n so that F < G, where F and G are Boolean 
functions if and only if G(x, x2,...,Xn) = 1 whenever 
F (x1, %2,...,%) = 1. 


Each of the sets {+-, ~} and {-, ~} is functionally complete. 

Each of the sets { | } and { | } is functionally complete. 

The use of K-maps to minimize Boolean expressions. 

The Quine—McCluskey method for minimizing Boolean ex- 
pressions. 


9. Is there a single type of logic gate that can be used to build 
all circuits that can be built using OR gates, AND gates, 
and inverters? 

10. a) Explain how K-maps can be used to simplify sum-of- 
products expansions in three Boolean variables. 
b) Use a K-map to simplify the sum-of-products expan- 
sionxyz+xyzt+txyz+xXyz+X yz. 
11. a) Explain how K-maps can be used to simplify sum-of- 
products expansions in four Boolean variables. 
b) Use a K-map to simplify the sum-of-products ex- 
pansion wxyz+wxyzZ +wxyz+wxyZ+wxyz+ 
WX yZ+Wxyz+wxyr+wxyz. 
12. a) What is a don’t care condition? 
b) Explain how don’t care conditions can be used to build 
a circuit using OR gates, AND gates, and inverters 
that produces an output of | if a decimal digit is 6 or 
greater, and an output of 0 if this digit is less than 6. 
13. a) Explain how to use the Quine—McCluskey method to 
simplify sum-of-products expansions. 
b) Use this method to simplify xyz+xyz+xyz+ 
Xyz+xyz. 


6. Determine whether F < G or G < F for the following 
pairs of functions. 
a) F(x, y)=x,G@,y)=x+y 
b) F@,y)=x+y, GQ, y) =xy 
c) F@,y) =%,G@,y)=x+y 

7. Show that if F and G are Boolean functions of degree n, 
then 
a) F< F+4+G. b) FG <F. 

8. Show that if F, G, and H are Boolean functions of degree 
n, then F+G < Hifandonlyif F < HandG < dH. 

*9, Show that the relation < is a partial ordering on the set of 

Boolean functions of degree n. 

*10. Draw the Hasse diagram for the poset consisting of the 
set of the 16 Boolean functions of degree two (shown in 
Table 3 of Section 12.1) with the partial ordering <. 

*11. For each of these equalities either prove it is an identity 
or find a set of values of the variables for which it does 
not hold. 
axlla=alylz 
bxloloa=@lylala 
Oxlylg=alwyl@lys 


Define the Boolean operator © as follows:1 © 1= 1,100= 
0,001=0,and000=1. 
12. Show thatx O y=xy+Xy. 
13. Show thatx Oy=(x* @y). 
14. Show that each of these identities holds. 
a)xOx=1 b) xOx=0 
ec) xOy=yOux 
15. Is it always true that (x © y)Oz=xO(y©z)? 
*16. Determine whether the set {©} is functionally complete. 


*17, How many of the 16 Boolean functions in two variables 
x and y can be represented using only the given set of 
operators, variables x and y, and values 0 and 1? 
at} bt} oc {+} dd {4+} 

The notation for an XOR gate, which produces the output 
x ® y from x and y, is as follows: 


18. Determine the output of each of these circuits. 
a) 
x 
>> 


> 
prep 
} >—— 


19. Show how a half adder can be constructed using fewer 
gates than are used in Figure 8 of Section 12.3 when XOR 
gates can be used in addition to OR gates, AND gates, and 
inverters. 


b) 


20. Design a circuit that determines whether three or more 
of four individuals on a committee vote yes on an issue, 
where each individual uses a switch for the voting. 


Computer Projects 


Write programs with these input and output. 


1. Given the values of two Boolean variables x and y, find 
the values of x + y,xy,x ®y,x | y,andx | y. 

2. Construct a table listing the set of values of all 256 
Boolean functions of degree three. 

3. Given the values of a Boolean function in n variables, 


where n is a positive integer, construct the sum-of- 
products expansion of this function. 
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q* threshold gate produces an output y that is either 0 


or 1 given a set of input values for the Boolean variables 
X1,X2,...,%X,. A threshold gate has a threshold value 7, 
which is a real number, and weights w1, w2,..., Wn, each of 
which is a real number. The output y of the threshold gate is 1 
if and only if wyx] + wox2 +---+ WX, => T. The threshold 
gate with threshold value T and weights w,, w2, ..., Wy iStep- 
resented by the following diagram. Threshold gates are useful 
in modeling in neurophysiology and in artificial intelligence. 


Xn 
21. A threshold gate represents a Boolean function. Find a 
Boolean expression for the Boolean function represented 
by this threshold gate. 


xy 


#3 


22. A Boolean function that can be represented by a thresh- 
old gate is called a threshold function. Show that each 
of these functions is a threshold function. 

a) F(x) =X b) FQ, y)=x+y 
c) F(x, y)=xy d) F(x, y)=x|y 
e) F(x,y)=x ly f) FQx,y,z=x+ yz 
g) Fw,x,y,z)=w+xytz 
h) F(w, x,y,z) =wxz+xyz 
* 23. Show that F(x, y) = x @ y is not a threshold function. 
*24. Show that F(w, x, y,z) =wx-+ yz is not a threshold 
function. 


4. Given the table of values of a Boolean function, express 
this function using only the operators: and . 


5. Given the table of values of a Boolean function, express 
this function using only the operators + and . 


*6. Given the table of values of a Boolean function, express 
this function using only the operator | . 
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Given the table of values of a Boolean function, express 
this function using only the operator |. 


Given the table of values of a Boolean function of degree 
three, construct its K-map. 


Given the table of values of a Boolean function of degree 
four, construct its K-map. 


Given the table of values of a Boolean function, use the 


Computations and Explorations 


11. 


12. 


Quine—McCluskey method to find a minimal sum-of- 
products representation of this function. 


Given a threshold value and a set of weights for a thresh- 
old gate and the values of the n Boolean variables in the 
input, determine the output of this gate. 


Given a positive integer n, construct a random Boolean 
expression in n variables in disjunctive normal form. 


Use a computational program or programs you have written to do these exercises. 


im) 


. Compute the number of Boolean functions of degrees 


seven, eight, nine, and ten. 


. Construct a table of the Boolean functions of degree three. 


. Construct a table of the Boolean functions of degree four. 


. Express each of the different Boolean expressions in three 


variables in disjunctive normal form with just the NAND 
operator, using as few NAND operators as possible. What 
is the largest number of NAND operators required? 


. Express each of the different Boolean expressions in dis- 


junctive normal form in four variables using just the NOR 


Writing Projects 


Respond to these with essays using outside sources. 


. Describe some of the early machines devised to solve 


problems in logic, such as the Stanhope Demonstrator, 
Jevons’s Logic Machine, and the Marquand Machine. 


. Explain the difference between combinational circuits 


and sequential circuits. Then explain how flip-flops are 
used to build sequential circuits. 


. Define a shift register and discuss how shift registers are 


used. Show how to build shift registers using flip-flops 
and logic gates. 


. Show how multipliers can be built using logic gates. 
. Find out how logic gates are physically constructed. Dis- 


cuss whether NAND and NOR gates are used in building 
circuits. 


. Explain how dependency notation can be used to describe 


complicated switching circuits. 


. Describe how multiplexers are used to build switching 


circuits. 


10. 


11. 


12. 


operator, with as few NOR operators as possible. What is 
the largest number of NOR operators required? 


. Randomly generate 10 different Boolean expressions in 


four variables and determine the average number of steps 
required to minimize them using the Quine—McCluskey 
method. 


. Randomly generate 10 different Boolean expressions in 


five variables and determine the average number of steps 
required to minimize them using the Quine—McCluskey 
method. 


. Explain the advantages of using threshold gates to con- 


struct switching circuits. Illustrate this by using threshold 
gates to construct half and full adders. 


. Describe the concept of hazard-free switching circuits and 


give some of the principles used in designing such circuits. 


Explain how to use K-maps to minimize functions of six 
variables. 


Discuss the ideas used by newer methods for minimiz- 
ing Boolean functions, such as Espresso. Explain how 
these methods can help solve minimization problems in 
as many as 25 variables. 


Describe what is meant by the functional decomposition 
of a Boolean function of 1 variables and discuss proce- 
dures for decomposing Boolean functions into a compo- 
sition of Boolean functions with fewer variables. 


13.1 Languages 
and 
Grammars 


13.2 Finite-State 
Machines 
with Output 


13.3 Finite-State 
Machines 
with No 
Output 


13.4 Language 
Recognition 


13.5 Turing 
Machines 


Modeling Computation 


C omputers can perform many tasks. Given a task, two questions arise. The first is: Can it 
be carried out using a computer? Once we know that this first question has an affirmative 
answer, we can ask the second question: How can the task be carried out? Models of computation 
are used to help answer these questions. 

We will study three types of structures used in models of computation, namely, grammars, 
finite-state machines, and Turing machines. Grammars are used to generate the words of a 
language and to determine whether a word is in a language. Formal languages, which are 
generated by grammars, provide models both for natural languages, such as English, and for 
programming languages, such as Pascal, Fortran, Prolog, C, and Java. In particular, grammars 
are extremely important in the construction and theory of compilers. The grammars that we will 
discuss were first used by the American linguist Noam Chomsky in the 1950s. 

Various types of finite-state machines are used in modeling. All finite-state machines have 
a set of states, including a starting state, an input alphabet, and a transition function that assigns 
a next state to every pair of a state and an input. The states of a finite-state machine give it 
limited memory capabilities. Some finite-state machines produce an output symbol for each 
transition; these machines can be used to model many kinds of machines, including vending 
machines, delay machines, binary adders, and language recognizers. We will also study finite- 
state machines that have no output but do have final states. Such machines are extensively used 
in language recognition. The strings that are recognized are those that take the starting state 
to a final state. The concepts of grammars and finite-state machines can be tied together. We 
will characterize those sets that are recognized by a finite-state machine and show that these are 
precisely the sets that are generated by a certain type of grammar. 

Finally, we will introduce the concept of a Turing machine. We will show how Turing 
machines can be used to recognize sets. We will also show how Turing machines can be used 
to compute number-theoretic functions. We will discuss the Church—Turing thesis, which states 
that every effective computation can be carried out using a Turing machine. We will explain 
how Turing machines can be used to study the difficulty of solving certain classes of problems. 
In particular, we will describe how Turing machines are used to classify problems as tractable 
versus intractable and solvable versus unsolvable. 


Languages and Grammars 


Introduction 


Words in the English language can be combined in various ways. The grammar of English tells 
us whether a combination of words is a valid sentence. For instance, the frog writes neatly is 
a valid sentence, because it is formed from a noun phrase, the frog, made up of the article the 
and the noun frog, followed by a verb phrase, writes neatly, made up of the verb writes and the 
adverb neatly. We do not care that this is a nonsensical statement, because we are concerned 
only with the syntax, or form, of the sentence, and not its semantics, or meaning. We also note 
that the combination of words swims quickly mathematics is not a valid sentence because it does 
not follow the rules of English grammar. 

The syntax of a natural language, that is, a spoken language, such as English, French, 
German, or Spanish, is extremely complicated. In fact, it does not seem possible to specify all 
the rules of syntax for a natural language. Research in the automatic translation of one language 
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to another has led to the concept of a formal language, which, unlike a natural language, is 
specified by a well-defined set of rules of syntax. Rules of syntax are important not only in 
linguistics, the study of natural languages, but also in the study of programming languages. 

We will describe the sentences of a formal language using a grammar. The use of grammars 
helps when we consider the two classes of problems that arise most frequently in applications 
to programming languages: (1) How can we determine whether a combination of words is a 
valid sentence in a formal language? (2) How can we generate the valid sentences of a formal 
language? Before giving a technical definition of a grammar, we will describe an example of 
a grammar that generates a subset of English. This subset of English is defined using a list of 
rules that describe how a valid sentence can be produced. We specify that 


— 


. asentence is made up of a noun phrase followed by a verb phrase; 


NO 


. anoun phrase is made up of an article followed by an adjective followed by a noun, 
or 


. anoun phrase is made up of an article followed by a noun; 
. averb phrase is made up of a verb followed by an adverb, or 
. a verb phrase is made up of a verb; 


3 

4 

5 

6. an article is a, or 
7. an article is the; 

8. an adjective is large, or 
9. an adjective is hungry; 
10. anoun is rabbit, or 

11. anoun is mathematician, 
12. a verb is eats, or 

13. a verb is hops; 

14. an adverb is quickly, or 


15. an adverb is wildly. 


From these rules we can form valid sentences using a series of replacements until no more rules 
can be used. For instance, we can follow the sequence of replacements: 


sentence 

noun phrase verb phrase 

article adjective noun verb phrase 
article adjective noun verb adverb 
the adjective noun verb adverb 

the largenoun verb adverb 

the large rabbit verb adverb 

the large rabbit hops adverb 

the large rabbit hops quickly 


to obtain a valid sentence. It is also easy to see that some other valid sentences are: a hungry 
mathematician eats wildly, a large mathematician hops, the rabbit eats quickly, and so on. Also, 
we can see that the quickly eats mathematician is not a valid sentence. 


Phrase-Structure Grammars 


Before we give a formal definition of a grammar, we introduce a little terminology. 


DEFINITION 1 


The notion of a 
phrase-structure 
grammar extends 

the concept of a 
rewrite system 
devised by Axel Thue 
in the early 20th 
century. 


DEFINITION 2 


EXAMPLE 1 


DEFINITION 3 
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A vocabulary (or alphabet) V is a finite, nonempty set of elements called symbols. A word (or 
sentence) over V is a string of finite length of elements of V. The empty string or null string, 
denoted by A, is the string containing no symbols. The set of all words over V is denoted 
by V*.A language over V is a subset of V*. 


Note that 4, the empty string, is the string containing no symbols. It is different from J, the 
empty set. It follows that {4} is the set containing exactly one string, namely, the empty string. 

Languages can be specified in various ways. One way is to list all the words in the language. 
Another is to give some criteria that a word must satisfy to be in the language. In this section, we 
describe another important way to specify a language, namely, through the use of a grammar, 
such as the set of rules we gave in the introduction to this section. A grammar provides a 
set of symbols of various types and a set of rules for producing words. More precisely, a 
grammar has a vocabulary V, which is a set of symbols used to derive members of the language. 
Some of the elements of the vocabulary cannot be replaced by other symbols. These are called 
terminals, and the other members of the vocabulary, which can be replaced by other symbols, 
are called nonterminals. The sets of terminals and nonterminals are usually denoted by T 
and JN, respectively. In the example given in the introduction of the section, the set of terminals 
is {a, the, rabbit, mathematician, hops, eats, quickly, wildly}, and the set of nonterminals is 
{sentence, noun phrase, verb phrase, adjective, article, noun, verb, adverb}. There is a 
special member of the vocabulary called the start symbol, denoted by S, which is the element 
of the vocabulary that we always begin with. In the example in the introduction, the start symbol 
is sentence. The rules that specify when we can replace a string from V%, the set of all strings 
of elements in the vocabulary, with another string are called the productions of the grammar. 
We denote by zo — z; the production that specifies that zo can be replaced by z; within a 
string. The productions in the grammar given in the introduction of this section were listed. 
The first production, written using this notation, is sentence — noun phrase verb phrase. We 
summarize this terminology in Definition 2. 


A phrase-structure grammar G = (V,T,S, P) consists of a vocabulary V, a subset T 
of V consisting of terminal symbols, a start symbol S from V, and a finite set of pro- 
ductions P. The set V — T is denoted by N. Elements of N are called nonterminal symbols. 
Every production in P must contain at least one nonterminal on its left side. 


Let G = (V,T, S, P), where V = {a,b, A, B, S}, T = {a, b}, S is the start symbol, and P = 
{S > ABa, A > BB, B — ab, AB — b}. G is an example of a phrase-structure grammar. 4 


We will be interested in the words that can be generated by the productions of a phrase- 
structure grammar. 


Let G = (V,T,S, P) be a phrase-structure grammar. Let wo = /zor (that is, the concate- 
nation of /, zo, and r) and w; = /z)r be strings over V. If zo > z, is a production of G, 
we say that w, is directly derivable from wo and we write wo > w1. If wo, W1,..., Wn are 
strings over V such that wo > w1, W] > W2, .--, Wn—1 > Wn, then we say that w, 1s derivable 
from wo, and we write wo + wy. The sequence of steps used to obtain w, from wo is called 
a derivation. 
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EXAMPLE 2 


DEFINITION 4 


EXAMPLE 3 


EXAMPLE 4 


EXAMPLE 5 


The string Aaba is directly derivable from ABa in the grammar in Example | because 
B — ab is a production in the grammar. The string abababa is derivable from ABa be- 
cause ABa = Aaba = BBaba = Bababa = abababa, using the productions B + ab, A > BB, 
B — ab, and B — ab in succession. 


Let G = (V, T, S, P) be a phrase-structure grammar. The language generated by G (or the 
language of G), denoted by L(G), is the set of all strings of terminals that are derivable from 
the starting state S. In other words, 


L(G) = {we T*|S4 wh. 


In Examples 3 and 4 we find the language generated by a phrase-structure grammar. 


Let G be the grammar with vocabulary V = {S, A, a, b}, set of terminals T = {a, b}, starting 
symbol S, and productions P = {S — aA, S — b, A > aa}. What is L(G), the language of 
this grammar? 


Solution: From the start state S we can derive aA using the production S —~ aA. We can 
also use the production S$ — b to derive b. From aA the production A — aa can be used to 
derive aaa. No additional words can be derived. Hence, L(G) = {b, aaa}. 


Let G be the grammar with vocabulary V = {S, 0, 1}, set of terminals T = {0, 1}, starting sym- 
bol S, and productions P = {S — 11S, S — 0}. What is L(G), the language of this grammar? 


Solution: From S we can derive 0 using S > 0, or 11S using S > 11S. From 11S we can 
derive either 110 or 1111S. From 1111S we can derive 11110 and 1111115. At any stage of a 
derivation we can either add two Is at the end of the string or terminate the derivation by adding 
a 0 at the end of the string. We surmise that L(G) = {0, 110, 11110, 1111110, ...}, the set of 
all strings that begin with an even number of Is and end with a 0. This can be proved using 
an inductive argument that shows that after n productions have been used, the only strings of 
terminals generated are those consisting of n — | concatenations of 11 followed by 0. (This is 
left as an exercise for the reader.) 


The problem of constructing a grammar that generates a given language often arises. Ex- 
amples 5, 6, and 7 describe problems of this kind. 


Give a phrase-structure grammar that generates the set {0"1” |n = 0, 1, 2,...}. 


Solution: Two productions can be used to generate all strings consisting of a string of Os followed 
by a string of the same number of Is, including the null string. The first builds up successively 
longer strings in the language by adding a 0 at the start of the string and a | at the end. The second 
production replaces S with the empty string. The solution is the grammar G = (V,T, S, P), 
where V = {0, 1, S}, T = {0, 1}, S is the starting symbol, and the productions are 


S—> 0S1 
Sd. 


The verification that this grammar generates the correct set is left as an exercise for the 
reader. 


EXAMPLE 6 


EXAMPLE 7 


Links 
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Example 5 involved the set of strings made up of Os followed by 1s, where the number 
of Os and Is are the same. Example 6 considers the set of strings consisting of Os followed 
by 1s, where the number of Os and Is may differ. 


Find a phrase-structure grammar to generate the set {0’"1” | m and n are nonnegative integers}. 


Solution: We will give two grammars G, and Go that generate this set. This will illustrate that 
two grammars can generate the same language. 

The grammar G has alphabet V = {S, 0, 1}; terminals T = {0, 1}; and productions S > 0S, 
S — S1, and S — i. G; generates the correct set, because using the first production m times 
puts m Os at the beginning of the string, and using the second production n times puts n Is at 
the end of the string. The details of this verification are left to the reader. 

The grammar G2 has alphabet V = {S, A, 0, 1}; terminals T = {0, 1}; and productions 
S—0S,S—> 1A,S—- 1, A— 1A, A > 1, and S —> 4. The details that this grammar gen- 
erates the correct set are left as an exercise for the reader. < 


Sometimes a set that is easy to describe can be generated only by a complicated grammar. 
Example 7 illustrates this. 


One grammar that generates the set {0"1"2" |n=0,1,2,3,...} is G=(V,T,S, P) 
with V = {0,1,2,5, A,B,C}; T = {0,1,2}; starting state S$; and productions $ > C, 
C — OCAB, S — i, BA > AB, 0A > O01, 1A => 11, 1B > 12, and 2B — 22. We leave 
it as an exercise for the reader (Exercise 12) to show that this statement is correct. The grammar 
given is the simplest type of grammar that generates this set, in a sense that will be made clear 
later in this section. 


Types of Phrase-Structure Grammars 


Phrase-structure grammars can be classified according to the types of productions that are al- 
lowed. We will describe the classification scheme introduced by Noam Chomsky. In Section 13.4 
we will see that the different types of languages defined in this scheme correspond to the classes 
of languages that can be recognized using different models of computing machines. 

A type 0 grammar has no restrictions on its productions. A type 1 grammar can have 
productions of the form w; — w2, where w; = /Ar and w2 = Iwr, where A is a nonterminal 
symbol, / andr are strings of zero or more terminal or nonterminal symbols, and w is anonempty 
string of terminal or nonterminal symbols. It can also have the production S — A as long 
as S does not appear on the right-hand side of any other production. A type 2 grammar can 
have productions only of the form w; — w2, where w is a single symbol that is not a terminal 
symbol. A type 3 grammar can have productions only of the form w; — w2 with w; = A and 
either w2 = aB or w2 = a, where A and B are nonterminal symbols and a is a terminal symbol, 
or with w; = S and w2 = 2d. 

Type 2 grammars are called context-free grammars because a nonterminal symbol that is 
the left side of a production can be replaced in a string whenever it occurs, no matter what else is 
in the string. A language generated by a type 2 grammar is called a context-free language. When 
there is a production of the form /w}r — lwar (but not of the form w; — w2), the grammar is 
called type | or context-sensitive because w; can be replaced by w2 only when it is surrounded 
by the strings / and r. A language generated by a type | grammar is called a context-sensitive 
language. Type 3 grammars are also called regular grammars. A language generated by a 
regular grammar is called regular. Section 13.4 deals with the relationship between regular 
languages and finite-state machines. 

Of the four types of grammars we have defined, context-sensitive grammars have the most 
complicated definition. Sometimes, these grammars are defined in a different way. A production 
of the form w; — wz is called noncontracting if the length of w, is less than or equal to the 
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EXAMPLE 9 


EXAMPLE 10 


length of w2. According to our characterization of context-senstive languages, every production 
inatype | grammar, other than the production S — 4, if itis present, is noncontracting. It follows 
that the lengths of the strings in a derivation in a context-sensitive language are nondecreasing 
unless the production S — A is used. This means that the only way for the empty string to belong 
to the language generated by a context-sensitive grammar is for the production S — i to be 
part of the grammar. The other way that context-sensitive grammars are defined is by specifying 
that all productions are noncontracting. A grammar with this property is called noncontracting 
or monotonic. The class of noncontracting grammars is not the same as the class of context- 
sensitive grammars. However, these two classes are closely related; it can be shown that they 
define the same set of languages except that noncontracting grammars cannot generate any 
language containing the empty string A. 


From Example 6 we know that {0’" 1” | m,n = 0, 1, 2,...}is aregular language, because it can 
be generated by a regular grammar, namely, the grammar G2 in Example 6. < 


Context-free and regular grammars play an important role in programming languages. 
Context-free grammars are used to define the syntax of almost all programming languages. 
These grammars are strong enough to define a wide range of languages. Furthermore, efficient 
algorithms can be devised to determine whether and how a string can be generated. Regular 
grammars are used to search text for certain patterns and in lexical analysis, which is the process 
of transforming an input stream into a stream of tokens for use by a parser. 


It follows from Example 5 that {0"1” | n = 0,1, 2, ...} is a context-free language, because the 
productions in this grammar are S — OS1 and S — A. However, it is not a regular language. 
This will be shown in Section 13.4. 


The set {0"1"2” |n = 0,1, 2,...} is a context-sensitive language, because it can be generated 
by a type | grammar, as Example 7 shows, but not by any type 2 language. (This is shown in 
Exercise 28 in the supplementary exercises at the end of the chapter.) < 


Table 1 summarizes the terminology used to classify phrase-structure grammars. 


Derivation Trees 


A derivation in the language generated by a context-free grammar can be represented graphically 
using an ordered rooted tree, called a derivation, or parse tree. The root of this tree represents 
the starting symbol. The internal vertices of the tree represent the nonterminal symbols that 
arise in the derivation. The leaves of the tree represent the terminal symbols that arise. If the 
production A — w arises in the derivation, where w is a word, the vertex that represents A has 
as children vertices that represent each symbol in w, in order from left to right. 


TABLE 1 Types of Grammars. 


Restrictions on Productions w4 > w2 


0 No restrictions 
1 w, =/Ar and w2 = lwr, where A € N,1,r, w € (NUT)* andw 4A; 
or w; = S and w2 = i as long as S is not on the right-hand side of another production 
2 w1 = A, where A is a nonterminal symbol 
3 w, = A and w2 = aB or w2 = a, where Ae N, Be N, anda € T; orw; = S andw2 =A 


EXAMPLE 11 


EXAMPLE 12 
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sentence 
a 
noun phrase verb phrase 
PA FS 
article adjective noun verb adverb 
the hungry rabbit eats quickly 


FIGURE 1 A Derivation Tree. 


Construct a derivation tree for the derivation of the hungry rabbit eats quickly, given in the 
introduction of this section. 


Solution: The derivation tree is shown in Figure 1. < 


The problem of determining whether a string is in the language generated by a context-free 
grammar arises in many applications, such as in the construction of compilers. Two approaches 
to this problem are indicated in Example 12. 


Determine whether the word cbab belongs to the language generated by the grammar G = 
(V,T,S, P), where V = {a,b,c, A, B, C, S}, T = {a, b, c}, S is the starting symbol, and the 
productions are 


S — AB 
A — Ca 
B—- Ba 
B—+>Cb 
Bb 

C > cb 
C => b. 


Solution: One way to approach this problem is to begin with S and attempt to derive cbab using a 
series of productions. Because there is only one production with S on its left-hand side, we must 
start with § + AB. Next we use the only production that has A on its left-hand side, namely, 
A — Ca, to obtain $+ AB = CaB. Because cbhab begins with the symbols cb, we use the 
production C — cb. This gives us § = AB > CaB = cbaB. We finish by using the production 
B — b, to obtain S + AB = CaB = cbaB = cbab. The approach that we have used is called 
top-down parsing, because it begins with the starting symbol and proceeds by successively 
applying productions. 

There is another approach to this problem, called bottom-up parsing. In this approach, we 
work backward. Because cbab is the string to be derived, we can use the production C — cb, so 


AVRAM NOAM CHOMSKY (BORN 1928) Noam Chomsky, born in Philadelphia, is the son of a Hebrew 
scholar. He received his B.A., M.A., and Ph.D. in linguistics, all from the University of Pennsylvania. He was 
on the staff of the University of Pennsylvania from 1950 until 1951. In 1955 he joined the faculty at M.LT., 
beginning his M.I.T. career teaching engineers French and German. Chomsky is currently the Ferrari P. Ward 
Professor of foreign languages and linguistics at M.I.T. He is known for his many fundamental contributions 
to linguistics, including the study of grammars. Chomsky is also widely known for his outspoken political 
activism. 
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Links 


The ancient Indian 
grammarian Panini 
specified Sanskrit using 
3959 rules; Backus-Naur 
form is sometimes called 
Backus-Panini form. 


that Cab = cbab. Then, we can use the production A + Ca, so that Ab = Cab = cbab. Using the 
production B — b gives AB = Ab = Cab = cbab. Finally, using $ — AB shows that a complete 
derivation for cbab is S = AB = Ab = Cab = cbab. < 


Backus—Naur Form 


There is another notation that is sometimes used to specify a type 2 grammar, called the 
Backus-Naur form (BNF), after John Backus, who invented it, and Peter Naur, who refined 
it for use in the specification of the programming language ALGOL. (Surprisingly, a notation 
quite similar to the Backus-Naur form was used approximately 2500 years ago to describe 
the grammar of Sanskrit.) The Backus—Naur form is used to specify the syntactic rules of 
many computer languages, including Java. The productions in a type 2 grammar have a single 
nonterminal symbol as their left-hand side. Instead of listing all the productions separately, 
we can combine all those with the same nonterminal symbol on the left-hand side into one 


statement. Instead of using the symbol — in a production, we use the symbol ::=. We enclose 
all nonterminal symbols in brackets, (), and we list all the right-hand sides of productions in 
the same statement, separating them by bars. For instance, the productions A > Aa, A > a, 
and A — AB can be combined into (A) ::= (A)a | a | (A)(B). 

Example 13 illustrates how the Backus—Naur form is used to describe the syntax of pro- 
gramming languages. Our example comes from the original use of Backus—Naur form in the 
description of ALGOL 60. 


JOHN BACKUS (1924—2007) John Backus was born in Philadelphia and grew up in Wilmington, Delaware. 
He attended the Hill School in Pottstown, Pennsylvania. He needed to attend summer school every year because 
he disliked studying and was not a serious student. But he enjoyed spending his summers in New Hampshire 
where he attended summer school and amused himself with summer activities, including sailing. He obliged 
his father by enrolling at the University of Virginia to study chemistry. But he quickly decided chemistry was 
not for him, and in 1943 he entered the army, where he received medical training and worked in a neurosurgery 
7 ward in an army hospital. Ironically, Backus was soon diagnosed with a bone tumor in his skull and was fitted 
with a metal plate. His medical work in the army convinced him to try medical school, but he abandoned this 
after nine months because he disliked the rote memorization required. After dropping out of medical school, 
he entered a school for radio technicians because he wanted to build his own high fidelity set. A teacher in this school recognized his 
potential and asked him to help with some mathematical calculations needed for an article in a magazine. Finally, Backus found what 
he was interested in: mathematics and its applications. He enrolled at Columbia University, from which he received both bachelor’s 
and master’s degrees in mathematics. Backus joined IBM as a programmer in 1950. He participated in the design and development 
of two of IBM’s early computers. From 1954 to 1958 he led the IBM group that developed FORTRAN. Backus became a staff 
member at the IBM Watson Research Center in 1958. He was part of the committees that designed the programming language 
ALGOL, using what is now called the Backus—Naur form for the description of the syntax of this language. Later, Backus worked 
on the mathematics of families of sets and on a functional style of programming. Backus became an IBM Fellow in 1963, and he 
received the National Medal of Science in 1974 and the prestigious Turing Award from the Association of Computing Machinery in 
1977. 


PETER NAUR (BORN 1928) Peter Naur was born in Frederiksberg, near Copenhagen. As a boy he became 
interested in astronomy. Not only did he observe heavenly bodies, but he also computed the orbits of comets 
and asteroids. Naur attended Copenhagen University, receiving his degree in 1949. He spent 1950 and 1951 in 
Cambridge, where he used an early computer to calculate the motions of comets and planets. After returning to 
Denmark he continued working in astronomy but kept his ties to computing. In 1955 he served as a consultant 
to the building of the first Danish computer. In 1959 Naur made the switch from astronomy to computing as 
a full-time activity. His first job as a full-time computer scientist was participating in the development of the 
programming language ALGOL. From 1960 to 1967 he worked on the development of compilers for ALGOL 
and COBOL. In 1969 he became professor of computer science at Copenhagen University, where he has worked 
in the area of programming methodology. His research interests include the design, structure, and performance of computer programs. 
Naur has been a pioneer in both the areas of software architecture and software engineering. He rejects the view that computer 
programming is a branch of mathematics and prefers that computer science be called datalogy. 
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In ALGOL 60 an identifier (which is the name of an entity such as a variable) consists of a string 
of alphanumeric characters (that is, letters and digits) and must begin with a letter. We can use 


Pitre <") these rules in Backus—Naur to describe the set of allowable identifiers: 
(identifier) ::= (letter) | (identifier) (letter) | (identifier) (digit) 
(letter) :=a\|b\|---|y|z__ the ellipsis indicates that all 26 letters are included 
(digit) :=0O|1/2|3|/4|5|6|7|8|9 
For example, we can produce the valid identifier x99a by using the first rule to replace (identifier) 
by (identifier) (letter), the second rule to obtain (identifier)a, the first rule twice to obtain 
(identifier) (digit) (digit)a, the third rule twice to obtain (identifier) 99a, the first rule to obtain 
(letter)99a, and finally the second rule to obtain «99a. < 
EXAMPLE 14 _ What is the Backus—Naur form of the grammar for the subset of English described in the 
introduction to this section? 
Solution: The Backus—Naur form of this grammar is 
(sentence) ::= (noun phrase) (verb phrase) 
(noun phrase) ::= (article) (adjective) (noun) | (article) (noun) 
(verb phrase) ::= (verb) (adverb) | (verb) 
(article) ::= a | the 
(adjective) ::= large | hungry 
(noun) ::= rabbit | mathematician 
(verb) ::= eats | hops 4 
(adverb) ::= quickly | wildly 
EXAMPLE 15 _ Give the Backus—Naur form for the production of signed integers in decimal notation. (A signed 
integer is a nonnegative integer preceded by a plus sign or a minus sign.) 
Solution: The Backus—Naur form for a grammar that produces signed integers is 
(signed integer) ::= (sign) (integer) 
(sign) := + | —- 
(integer) ::= (digit) | (digit) (integer) 4 
(digit) :=0O|1/2|3/4|5|/6|7/8|9 
The Backus—Naur form, with a variety of extensions, is used extensively to specify the 
syntax of programming languages, such as Java and LISP; database languages, such as SQL; 
and markup languages, such as XML. Some extensions of the Backus—Naur form that are 
commonly used in the description of programming languages are introduced in the preamble to 
Exercise 34. 
Exercises 
Exercises 1—3 refer to the grammar with start symbol sen- sentence — noun phrase intransitive verb phrase 
tence, set of terminals T = {the, sleepy, happy, tortoise, hare, noun phrase — article adjective noun 
passes, runs, quickly, slowly}, set of nonterminals N = {noun noun phrase — article noun 
phrase, transitive verb phrase, intransitive verb phrase, transitive verb phrase — transitive verb 


article, adjective, noun, verb, adverb}, and productions: 


intransitive verb phrase — intransitive verb adverb 


sentence — noun phrase __ transitive verb phrase intransitive verb phrase — intransitive verb 


noun phrase 


article —> the 
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adjective — sleepy 
adjective — happy 

noun — tortoise 

noun —> hare 

transitive verb — passes 
intransitive verb — runs 
adverb — quickly 
adverb — slowly 


. Use the set of productions to show that each of these sen- 


tences is a valid sentence. 

a) the happy hare runs 

b) the sleepy tortoise runs quickly 

c) the tortoise passes the hare 

d) the sleepy hare passes the happy tortoise 


. Find five other valid sentences, besides those given in 


Exercise |. 


. Show that the hare runs the sleepy tortoise is not a valid 


sentence. 


. Let G=(V,T,S, P) be the phrase-structure grammar 


with V = {0,1, A, S}, T = {0, 1}, and set of produc- 

tions P consisting of S—> 1S, S > 00A, A — OA, 

and A — 0. 

a) Show that 111000 belongs to the language generated 
by G. 

b) Show that 11001 does not belong to the language gen- 
erated by G. 

c) What is the language generated by G? 


. Let G=(V,T,S, P) be the phrase-structure grammar 


with V = {0,1, A, B, S}, T = {0, 1}, and set of pro- 

ductions P consisting of S > 0A, S > 1A, A > OB, 

Bo>1A,B—>1. 

a) Show that 10101 belongs to the language generated 
by G. 

b) Show that 10110 does not belong to the language gen- 
erated by G. 

c) What is the language generated by G? 

Let V = {S, A, B,a, b} and T = {a, b}. Find the lan- 

guage generated by the grammar (V, 7, S, P) when the 

set P of productions consists of 

a) S—> AB, A — ab, B > bb. 

b) S—> AB,S—> aA, A— a, B => ba. 

c) S> AB,S > AA, A> aB,A->ab,B— b. 

d) S> AA, S> B, A> aaA, A— aa, B > bB, 
B= b. 

e) S> AB, A-— aAb, B > bBa,A>d, B= i. 


. Construct a derivation of 0713 using the grammar given 


in Example 5. 


. Show that the grammar given in Example 5 generates the 


set {0"1" |n =0, 1,2,...}. 


. a) Construct a derivation of 0714 using the grammar G, 


in Example 6. 

b) Construct a derivation of 0714 using the grammar G2 
in Example 6. 

a) Show that the grammar G, given in Example 6 gen- 
erates the set {0 1” | m,n =0,1,2,...}. 


11. 


*12, 


13. 


14. 


*15, 


16. 


17. 


18. 


19. 


b) Show that the grammar G2 in Example 6 generates 
the same set. 

Construct a derivation of 07172 in the grammar given in 

Example 7. 


Show that the grammar given in Example 7 generates the 

set {07172” |n =0,1,2,...}. 

Find a phrase-structure grammar for each of these lan- 

guages. 

a) the set consisting of the bit strings 0, 1, and 11 

b) the set of bit strings containing only Is 

c) the set of bit strings that start with 0 and end with 1 

d) the set of bit strings that consist of a 0 followed by an 
even number of Is 

Find a phrase-structure grammar for each of these lan- 

guages. 

a) the set consisting of the bit strings 10, 01, and 101 

b) the set of bit strings that start with 00 and end with 
one or more ls 

c) the set of bit strings consisting of an even number 
of 1s followed by a final 0 

d) the set of bit strings that have neither two consecutive 
Os nor two consecutive 1s 


Find a phrase-structure grammar for each of these lan- 

guages. 

a) the set of all bit strings containing an even number 
of Os and no Is 

b) the set of all bit strings made up of a 1 followed by an 
odd number of Os 

c) the set of all bit strings containing an even number 
of Os and an even number of 1s 

d) the set of all strings containing 10 or more Os and no 
Is 

e) the set of all strings containing more Os than 1s 

f) the set of all strings containing an equal number of Os 
and Is 

g) the set of all strings containing an unequal number 
of Os and Is 

Construct phrase-structure grammars to generate each of 

these sets. 

a) {1" |n = 0} 

ce) {(11)” |n = 0} 

Construct phrase-structure grammars to generate each of 

these sets. 

a) {0" |n = 0} 

c) {(000)" | n = 0} 

Construct phrase-structure grammars to generate each of 

these sets. 

a) {017" |n = 0} 

b) {0712" |n = 0} 

ce) {0"1"0" | m > Oandn > 0} 

Let V ={S,A,B,a,b} and T = {a,b}. Determine 

whether G = (V, 7, S, P) is a type 0 grammar but not 

a type 1 grammar, a type 1 grammar but not a type 2 

grammar, or a type 2 grammar but not a type 3 grammar 

if P, the set of productions, is 


a) S > aAB,A— Bb, B— 2. 


b) {10” | n > 0} 


b) {170 | n> 0} 


b) S> aA, A>a,A->b. 
c) S— ABa, AB > a. 
d) S — ABA, A — aB, B — ab. 


e) S> bDA,A> B,B- <a. 

f) S—aA,aA > B,B—>aA,A-— b. 
g) S>bA,A>b,S—>d. 

h) S — AB, B — aAb, aAb = b. 

i) S>aA,A—> bDB,B>b,B—>>X. 
jp S- A, A> B,B->d. 


20. A palindrome is a string that reads the same backward 
as it does forward, that is, a string w, where w = wk, 
where w* is the reversal of the string w. Find a context- 
free grammar that generates the set of all palindromes 
over the alphabet {0, 1}. 

*21. Let G; and G2 be context-free grammars, generating 
the languages L(G,) and L(G), respectively. Show 


that there is a context-free grammar generating each of 


these sets. 
a) L(G;)UL(G2) b) L(Gi)L(G2) 
c) L(G,)* 
22. Find the strings constructed using the derivation trees 
shown here. 


sentence 


a 


noun phrase verb phrase 


Fe 7 


article adjective noun verb adverb 
a large mathematician hops wildly 
signed integer 
a ~ 
sign integer 
a \ 
ag digit integer 
PN 
9 digit integer 
| 
8 digit 
7 


23. Construct derivation trees for the sentences in Exercise 1. 

24. Let G be the grammar with V = {a,b,c,S}; T= 
{a, b, c}; starting symbol S; and productions S — abS, 
S — bcS, S > bbS, S > a, and S — cb. Construct 
derivation trees for 


a) bcbba. 
ce) bcabbbbbcb. 


b) bbbcbba. 


*25, 


* 26. 


27. 


28. 


29. 


30. 


31. 


32. 
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Use top-down parsing to determine whether each of the 
following strings belongs to the language generated by 
the grammar in Example 12. 

a) baba b) abab 

c) cbaba d) bbbcba 

Use bottom-up parsing to determine whether the strings 
in Exercise 25 belong to the language generated by the 
grammar in Example 12. 


Construct a derivation tree for —109 using the grammar 
given in Example 15. 


a) Explain what the productions are in a grammar if the 
Backus—Naur form for productions is as follows: 


(expression) ::= ((expression)) | 
(expression) + (expression) | 
(expression) * (expression) | 
(variable) 

(variable) =x \|y 


b) Find a derivation tree for (x *« y) + x in this grammar. 


a) Construct a phrase-structure grammar that generates 
all signed decimal numbers, consisting of a sign, ei- 
ther + or —; a nonnegative integer; and a decimal 
fraction that is either the empty string or a decimal 
point followed by a positive integer, where initial ze- 
ros in an integer are allowed. 


b) Give the Backus—Naur form of this grammar. 

c) Construct a derivation tree for —31.4 in this grammar. 

a) Construct a phrase-structure grammar for the set of all 
fractions of the form a/b, where a is a signed integer 
in decimal notation and b is a positive integer. 

b) What is the Backus—Naur form for this grammar? 

c) Construct a derivation tree for +311/17 in this gram- 
mar. 

Give production rules in Backus—Naur form for an iden- 

tifier if it can consist of 

a) one or more lowercase letters. 

b) at least three but no more than six lowercase letters. 

c) one to six uppercase or lowercase letters beginning 
with an uppercase letter. 


d) a lowercase letter, followed by a digit or an under- 
score, followed by three or four alphanumeric char- 
acters (lower or uppercase letters and digits). 


Give production rules in Backus—Naur form for the name 
of a person if this name consists of a first name, which is 
a string of letters, where only the first letter is uppercase; 
a middle initial; and a last name, which can be any string 
of letters. 


. Give production rules in Backus—Naur form that gener- 


ate all identifiers in the C programming language. In C 
an identifier starts with a letter or an underscore (_) that 
is followed by one or more lowercase letters, uppercase 
letters, underscores, and digits. 
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Several extensions to Backus—Naur form are commonly used 
to define phrase-structure grammars. In one such extension, a 
question mark (?) indicates that the symbol, or group of sym- 
bols inside parentheses, to its left can appear zero or once (that 
is, it is optional), an asterisk (*) indicates that the symbol to 
its left can appear zero or more times, and a plus (+) indi- 
cates that the symbol to its left can appear one or more times. 
These extensions are part of extended Backus—Naur form 
(EBNF), and the symbols ?, *, and + are called metacharac- 
ters. In EBNF the brackets used to denote nonterminals are 
usually not shown. 

34. Describe the set of strings defined by each of these sets 

of productions in EBNF. 


a) string := L+D?L+ 


Luz=a|bie 
D:=0{|1 

b) string ::= sign D+ | D+ 
sign = + | -— 


D:=0/1/2/31/4|5|6|7/8|9 
¢c) string := L*(D+)?L* 

Lu=x\|y 

D:=0|1 

35. Give production rules in extended Backus—Naur form that 
generate all decimal numerals consisting of an optional 
sign, a nonnegative integer, and a decimal fraction that 
is either the empty string or a decimal point followed by 
an optional positive integer optionally preceded by some 
number of zeros. 

36. Give production rules in extended Backus—Naur form that 
generate a sandwich if a sandwich consists of a lower slice 
of bread; mustard or mayonnaise; optional lettuce; an op- 
tional slice of tomato; one or more slices of either turkey, 
chicken, or roast beef (in any combination); optionally 
some number of slices of cheese; and a top slice of bread. 

37. Give production rules in extended Backus—Naur form for 
identifiers in the C programming language (see Exer- 
cise 33). 


38. Describe how productions for a grammar in extended 
Backus—Naur form can be translated into a set of pro- 
ductions for the grammar in Backus—Naur form. 


This is the Backus—Naur form that describes the syntax of 
expressions in postfix (or reverse Polish) notation. 
expression) :: 


addOperator) :: 


= (term) | (term) (term) (addOperator) 
at |e 

= (factor) | (factor) (factor) (mulOperator) 
=*|/ 

(factor) ::= (identifier) | (expression) 

(identifier) :=a|b|---|z 


term) :: 


( 
( 
( 
(mulOperator) :: 
39. For each of these strings, determine whether it is gener- 


ated by the grammar given for postfix notation. If it is, 
find the steps used to generate the string 


a) abcx+ b) xy++ 
c) xy—Z* d) wxyz—*/ 
e) ade—x 


40. Use Backus—Naur form to describe the syntax of expres- 
sions in infix notation, where the set of operators and 
identifiers is the same as in the BNF for postfix expres- 
sions given in the preamble to Exercise 39, but parenthe- 
ses must surround expressions being used as factors. 

41. For each of these strings, determine whether it is gener- 
ated by the grammar for infix expressions from Exercise 
40. If it is, find the steps used to generate the string. 

a) x+y+z b) a/b+c/d 
c) mx(n+p) d) +m—n+p-—q 
e) (m+n) *(p—q) 

42. Let G be a grammar and let R be the relation contain- 
ing the ordered pair (wo, w}) if and only if w is directly 
derivable from wo in G. What is the reflexive transitive 
closure of R? 


Finite-State Machines with Output 


Introduction 


Links 


Many kinds of machines, including components in computers, can be modeled using a structure 
called a finite-state machine. Several types of finite-state machines are commonly used in models. 


All these versions of finite-state machines include a finite set of states, with a designated starting 
state, an input alphabet, and a transition function that assigns a next state to every state and input 
pair. Finite-state machines are used extensively in applications in computer science and data 
networking. For example, finite-state machines are the basis for programs for spell checking, 
grammar checking, indexing or searching large bodies of text, recognizing speech, transforming 
text using markup languages such as XML and HTML, and network protocols that specify how 


computers communicate. 


In this section, we will study those finite-state machines that produce output. We will show 
how finite-state machines can be used to model a vending machine, a machine that delays input, 
a machine that adds integers, and a machine that determines whether a bit string contains a 


specified pattern. 


Finite-state machines with 
output are often called 
finite-state transducers. 


DEFINITION 1 
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TABLE I State Table for a Vending Machine. 


Input Input 
State 5 10 25 O R 5 10 25 O R 
1 SQ S5 SO SO n n n n n 


SO K 
S| K 83 56 S| 
ky) S4 56 52 
83 S5 S56 83 
S4 56 56 S4 
S5 K 56 S56 S5 
S6 K 56 S6 AY) 


Before giving formal definitions, we will show how a vending machine can be modeled. A 
vending machine accepts nickels (5 cents), dimes (10 cents), and quarters (25 cents). When a 
total of 30 cents or more has been deposited, the machine immediately returns the amount in 
excess of 30 cents. When 30 cents has been deposited and any excess refunded, the customer 
can push an orange button and receive an orange juice or push a red button and receive an 
apple juice. We can describe how the machine works by specifying its states, how it changes 
states when input is received, and the output that is produced for every combination of input 
and current state. 

The machine can be in any of seven different states s;,i = 0, 1,2,...,6, where s; is the 
state where the machine has collected 5i cents. The machine starts in state so, with 0 cents 
received. The possible inputs are 5 cents, 10 cents, 25 cents, the orange button (O), and the red 
button (R). The possible outputs are nothing (7), 5 cents, 10 cents, 15 cents, 20 cents, 25 cents, 
an orange juice, and an apple juice. 

We illustrate how this model of the machine works with this example. Suppose that a student 
puts in a dime followed by a quarter, receives 5 cents back, and then pushes the orange button 
for an orange juice. The machine starts in state so. The first input is 10 cents, which changes 
the state of the machine to s2 and gives no output. The second input is 25 cents. This changes 
the state from s2 to sg, and gives 5 cents as output. The next input is the orange button, which 
changes the state from s6 back to so (because the machine returns to the start state) and gives 
an orange juice as its output. 

We can display all the state changes and output of this machine in a table. To do this we 
need to specify for each combination of state and input the next state and the output obtained. 
Table 1 shows the transitions and outputs for each pair of a state and an input. 

Another way to show the actions of a machine is to use a directed graph with labeled edges, 
where each state is represented by a circle, edges represent the transitions, and edges are labeled 
with the input and the output for that transition. Figure | shows such a directed graph for the 
vending machine. 


Finite-State Machines with Outputs 


We will now give the formal definition of a finite-state machine with output. 


A finite-state machine M = (S, 1, O, f, g, 59) consists of a finite set S of states, a finite input 
alphabet I, a finite output alphabet O, a transition function f that assigns to each state and 
input pair a new state, an output function g that assigns to each state and input pair an output, 
and an initial state so. 
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EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


FIGURE 1 A Vending Machine. 


Let M = (S, 1, O, f, g, 59) be a finite-state machine. We can use a state table to represent the 
values of the transition function f and the output function g for all pairs of states and input. We 
previously constructed a state table for the vending machine discussed in the introduction to 
this section. 


The state table shown in Table 2 describes a finite-state machine with S = {so, 51, 52, 53}, 


I = {0, 1}, and O = {0, 1}. The values of the transition function f are displayed in the first 
two columns, and the values of the output function g are displayed in the last two columns. 4 


Another way to represent a finite-state machine is to use a state diagram, which is a directed 
graph with labeled edges. In this diagram, each state is represented by a circle. Arrows labeled 
with the input and output pair are shown for each transition. 


Construct the state diagram for the finite-state machine with the state table shown in Table 2. 


Solution: The state diagram for this machine is shown in Figure 2. < 


Construct the state table for the finite-state machine with the state diagram shown in Figure 3. 


Solution: The state table for this machine is shown in Table 3. < 


FIGURE 2. The State Diagram for the 
Finite-State Machine Shown in Table 2. 


Start 
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a 


Input Input 
State 0 1 0 1 
K 1 83 


S| ky) 
53 S4 
S| S0 
53 S4 


FIGURE 3 A Finite-State Machine. 


EXAMPLE 4 


EXAMPLE 5 


An input string takes the starting state through a sequence of states, as determined by the 
transition function. As we read the input string symbol by symbol (from left to right), each input 
symbol takes the machine from one state to another. Because each transition produces an output, 
an input string also produces an output string. 

Suppose that the input string is x = x,x2...x,. Then, reading this input takes the machine 
from state sp to state 5, where s; = f (So, x1), then to state s2, where sz = f (s1, x2), and so on, 
with s; = f(sj-1, x;) for j = 1,2,...,k, ending at state s, = f (sx_1, xx). This sequence of 
transitions produces an output string yj yo... yx, where y; = g(so, x1) is the output correspond- 
ing to the transition from so to 51, y2 = g(S1, X2) is the output corresponding to the transition 
from s; to s2, and so on. In general, yj = g(sj-1, xj) for j = 1,2,..., k. Hence, we can extend 
the definition of the output function g to input strings so that g(x) = y, where y is the output 
corresponding to the input string x. This notation is useful in many applications. 


Find the output string generated by the finite-state machine in Figure 3 if the input string 
is 1O1011. 


Solution: The output obtained is 001000. The successive states and outputs are shown in 
Table 4. < 


We can now look at some examples of useful finite-state machines. Examples 5, 6, and 7 
illustrate that the states of a finite-state machine give it limited memory capabilities. The states 
can be used to remember the properties of the symbols that have been read by the machine. 
However, because there are only finitely many different states, finite-state machines cannot be 
used for some important purposes. This will be illustrated in Section 13.4. 


An important element in many electronic devices is a unit-delay machine, which produces as 
output the input string delayed by a specified amount of time. How can a finite-state machine 
be constructed that delays an input string by one unit of time, that is, produces as output the bit 
string Ox,x2...xx—1 given the input bit string x,x2...x,? 


Solution: A delay machine can be constructed that has two possible inputs, namely, 0 and 1. The 
machine must have a start state s9. Because the machine has to remember whether the previous 
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EXAMPLE 6 


? 


EXAMPLE 7 


00, 0 


FIGURE 4 A Unit-Delay Machine. FIGURE 5 A Finite-State Machine for 
Addition. 


input was a 0 or a 1, two other states s; and sz are needed, where the machine is in state s if the 
previous input was | and in state s2 if the previous input was 0. An output of 0 is produced for 
the initial transition from so. Each transition from s; gives an output of 1, and each transition 
from s2 gives an output of 0. The output corresponding to the input of a string x; ...x, is the 
string that begins with 0, followed by x,, followed by x2,..., ending with x,_,. The state 
diagram for this machine is shown in Figure 4. < 


Produce a finite-state machine that adds two positive integers using their binary expansions. 


Solution: When (x, ...x1x9)2 and (yy... y1yo)2 are added, the following procedure (as de- 
scribed in Section 4.2) is followed. First, the bits x9 and yo are added, producing a sum bit zo 
and a carry bit co. This carry bit is either 0 or 1. Then, the bits x; and y; are added, together 
with the carry co. This gives a sum bit z; and a carry bit cy. This procedure is continued until 
the nth stage, where x,, y,, and the previous carry c,—; are added to produce the sum bit z, and 
the carry bit c,, which is equal to the sum bit z,)+1. 

A finite-state machine to carry out this addition can be constructed using just two states. 
For simplicity we assume that both the initial bits x, and y, are 0 (otherwise we have to make 
special arrangements concerning the sum bit z,+1). The start state so is used to remember that 
the previous carry is 0 (or for the addition of the rightmost bits). The other state, 51, is used to 
remember that the previous carry is 1. 

Because the inputs to the machine are pairs of bits, there are four possible inputs. We 
represent these possibilities by 00 (when both bits are 0), 01 (when the first bit is 0 and the 
second is 1), 10 (when the first bit is 1 and the second is 0), and 11 (when both bits are 1). 
The transitions and the outputs are constructed from the sum of the two bits represented by 
the input and the carry represented by the state. For instance, when the machine is in state s1 
and receives 01 as input, the next state is s; and the output is 0, because the sum that arises is 
0+ 1+ 1 = (10)2. The state diagram for this machine is shown in Figure 5. < 


In a certain coding scheme, when three consecutive 1s appear in a message, the receiver of the 
message knows that there has been a transmission error. Construct a finite-state machine that 
gives a | as its current output bit if and only if the last three bits received are all Is. 


Solution: Three states are needed in this machine. The start state so remembers that the previous 
input value, if it exists, was not a 1. The state s; remembers that the previous input was a 1, 
but the input before the previous input, if it exists, was not a 1. The state s2 remembers that the 
previous two inputs were Is. 

An input of | takes sg to s;, because now a 1, and not two consecutive Is, has been read; it 
takes s; to s2, because now two consecutive 1s have been read; and it takes s to itself, because 
at least two consecutive Is have been read. An input of 0 takes every state to so, because this 
breaks up any string of consecutive 1s. The output for the transition from s2 to itself when a | 


DEFINITION 2 


Exercises 
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FIGURE 6 A Finite-State Machine That Gives an Output of 1 
If and Only If the Input String Read So Far Ends with 111. 


is read is 1, because this combination of input and state shows that three consecutive Is have 
been read. All other outputs are 0. The state diagram of this machine is shown in Figure 6. 4 


The final output bit of the finite-state machine we constructed in Example 7 is 1 if and only 
if the input string ends with 111. Because of this, we say that this finite-state machine recognizes 
the set of bit strings that end with 111. This leads us to Definition 2. 


Let M = (S, I, O, f, g, so) be a finite-state machine and L C J*. We say that M recognizes 
(or accepts) L if an input string x belongs to L if and only if the last output bit produced by 
M when given x as input is a 1. 


TYPES OF FINITE-STATE MACHINES Many different kinds of finite-state machines have 
been developed to model computing machines. In this section we have given a definition of one 
type of finite-state machine. In the type of machine introduced in this section, outputs correspond 
to transitions between states. Machines of this type are known as Mealy machines, because 
they were first studied by G. H. Mealy in 1955. There is another important type of finite-state 
machine with output, where the output is determined only by the state. This type of finite-state 
machine is known as a Moore machine, because E. F. Moore introduced this type of machine 
in 1956. Moore machines are considered in a sequence of exercises. 

In Example 7 we showed how a Mealy machine can be used for language recognition. 
However, another type of finite-state machine, giving no output, is usually used for this purpose. 
Finite-state machines with no output, also known as finite-state automata, have a set of final 
states and recognize a string if and only if it takes the start state to a final state. We will study 
this type of finite-state machine in Section 13.3. 


1. Draw the state diagrams for the finite-state machines with 
these state tables. 
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2. Give the state tables for the finite-state machines with 
these state diagrams. 
a) 
Start 


b) 


Start ———+| 


c) 


Start 


3. Find the output generated from the input string 01110 for 
the finite-state machine with the state table in 


a) Exercise l(a). 
b) Exercise 1(b). 
c) Exercise l(c). 
4. Find the output generated from the input string 10001 for 
the finite-state machine with the state diagram in 
a) Exercise 2(a). 
b) Exercise 2(b). 
c) Exercise 2(c). 
5. Find the output for each of these input strings when given 
as input to the finite-state machine in Example 2. 
a) 0111 b) 11011011 c) 01010101010 


6. Find the output for each of these input strings when given 
as input to the finite-state machine in Example 3. 


a) 0000 b) 101010 c) 11011100010 


7. Construct a finite-state machine that models an old- 
fashioned soda machine that accepts nickels, dimes, and 
quarters. The soda machine accepts change until 35 cents 
has been put in. It gives change back for any amount 
greater than 35 cents. Then the customer can push but- 
tons to receive either a cola, a root beer, or a ginger ale. 


8. Construct a finite-state machine that models a newspa- 
per vending machine that has a door that can be opened 
only after either three dimes (and any number of other 
coins) or a quarter and a nickel (and any number of other 
coins) have been inserted. Once the door can be opened, 
the customer opens it and takes a paper, closing the door. 
No change is ever returned no matter how much extra 
money has been inserted. The next customer starts with 
no credit. 


9. Construct a finite-state machine that delays an input string 
two bits, giving 00 as the first two bits of output. 


10. Construct a finite-state machine that changes every other 
bit, starting with the second bit, of an input string, and 
leaves the other bits unchanged. 


11. Construct a finite-state machine for the log-on procedure 
for a computer, where the user logs on by entering a user 
identification number, which is considered to be a single 
input, and then a password, which is considered to be a 
single input. If the password is incorrect, the user is asked 
for the user identification number again. 


12. Construct a finite-state machine for a combination lock 
that contains numbers | through 40 and that opens only 
when the correct combination, 10 right, 8 second left, 37 
right, is entered. Each input is a triple consisting of anum- 
ber, the direction of the turn, and the number of times the 
lock is turned in that direction. 


13. Construct a finite-state machine for a toll machine that 
opens a gate after 25 cents, in nickels, dimes, or quar- 
ters, has been deposited. No change is given for overpay- 
ment, and no credit is given to the next driver when more 
than 25 cents has been deposited. 


14. Construct a finite-state machine for entering a security 
code into an automatic teller machine (ATM) that imple- 
ments these rules: A user enters a string of four digits, 
one digit at a time. If the user enters the correct four dig- 
its of the password, the ATM displays a welcome screen. 
When the user enters an incorrect string of four digits, the 
ATM displays a screen that informs the user that an incor- 
rect password was entered. If a user enters the incorrect 


password three times, the account islocked. 


15 


Construct a finite-state machine for a restricted telephone 
switching system that implements these rules. Only calls 
to the telephone numbers 0, 911, and the digit 1 followed 
by 10-digit telephone numbers that begin with 212, 800, 
866, 877, and 888 are sent to the network. All other strings 
of digits are blocked by the system and the user hears an 
error message. 


16. Construct a finite-state machine that gives an output of 
1 if the number of input symbols read so far is divisible 
by 3 and an output of 0 otherwise. 


17. Construct a finite-state machine that determines whether 
the input string has a | in the last position and a 0 in the 
third to the last position read so far. 

18. Construct a finite-state machine that determines whether 
the input string read so far ends in at least five consecutive 
Is. 

19. Construct a finite-state machine that determines whether 
the word computer has been read as the last eight char- 
acters in the input read so far, where the input can be any 
string of English letters. 

A Moore machine M = (S, /, O, f, g, 50) consists of a fi- 
nite set of states, an input alphabet J, an output alphabet O, 
a transition function f that assigns a next state to every pair 
of a state and an input, an output function g that assigns an 
output to every state, and a starting state so. A Moore machine 
can be represented either by a table listing the transitions for 
each pair of state and input and the outputs for each state, 
or by a state diagram that displays the states, the transitions 
between states, and the output for each state. In the diagram, 
transitions are indicated with arrows labeled with the input, 
and the outputs are shown next to the states. 

20. Construct the state diagram for the Moore machine with 
this state table. 
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21. Construct the state table for the Moore machine with the 
state diagram shown here. Each input string to a Moore 
machine M produces an output string. In particular, the 
output corresponding to the input string aja2...ax is 
the string g(so)g(s1)... g(sx), where s; = f(s;-1, a;) for 
i=1,2,...,k. 


Start 


22. Find the output string generated by the Moore machine 
in Exercise 20 with each of these input strings. 
a) 0101 b) 111111 ce) 11101110111 
23. Find the output string generated by the Moore ma- 


chine in Exercise 21 with each of the input strings in 
Exercise 22. 


24. Construct a Moore machine that gives an output of 1 
whenever the number of symbols in the input string read 
so far is divisible by 4 and an output of 0 otherwise. 


25. Construct a Moore machine that determines whether an 
input string contains an even or odd number of 1s. The 
machine should give | as output if an even number of 1s 
are in the string and 0 as output if an odd number of 1s 
are in the string. 


Finite-State Machines with No Output 


Introduction 


One of the most important applications of finite-state machines is in language recognition. 
links xd = This application plays a fundamental role in the design and construction of compilers for pro- 
gramming languages. In Section 13.2 we showed that a finite-state machine with output can be 

used to recognize a language, by giving an output of | when a string from the language has 

been read and a 0 otherwise. However, there are other types of finite-state machines that are 

specially designed for recognizing languages. Instead of producing output, these machines have 

final states. A string is recognized if and only if it takes the starting state to one of these final 


States. 


Set of Strings 


Before discussing finite-state machines with no output, we will introduce some important back- 
ground material on sets of strings. The operations that will be defined here will be used exten- 
sively in our discussion of language recognition by finite-state machines. 
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DEFINITION 1 


EXAMPLE 1 


EXAMPLE 2 


DEFINITION 2 


EXAMPLE 3 


Suppose that A and B are subsets of V*, where V is a vocabulary. The concatenation of A 
and B, denoted by AB, is the set of all strings of the form xy, where x is a string in A and y 
is a string in B. 


Let A = {0, 11} and B = {1, 10, 110}. Find AB and BA. 


Solution: The set AB contains every concatenation of a string in A and a string in B. Hence, 
AB = {01, 010, 0110, 111, 1110, 11110}. The set BA contains every concatenation of a string 
in B and a string in A. Hence, BA = {10, 111, 100, 1011, 1100, 11011}. < 


Note that it is not necessarily the case that AB = BA when A and B are subsets of V%, 
where V is an alphabet, as Example | illustrates. 

From the definition of the concatenation of two sets of strings, we can define A”, for 
n=0,1,2,.... This is done recursively by specifying that 


A? = {a}, 
A"tl= A"A forn=0,1,2,.... 


Let A = {1,00}. Find A” for n = 0, 1, 2, and 3. 


Solution: We have A® = {A} and A! = A°A = {A}A = {1, 00}. To find A? we take concate- 
nations of pairs of elements of A. This gives A= {11, 100, 001, 0000}. To find A> we 
take concatenations of elements in A? and A; this gives A= {111, 1100, 1001, 10000, 
0011, 00100, 00001, 000000}. < 


Suppose that A is a subset of V*. Then the Kleene closure of A, denoted by A%, is the set 
consisting of concatenations of arbitrarily many strings from A. That is, A* = U2 AX, 


What are the Kleene closures of the sets A = {0}, B = {0, 1}, and C = {11}? 


Solution: The Kleene closure of A is the concatenation of the string 0 with itself an arbitrary 
finite number of times. Hence, A* = {0” |n = 0, 1,2,...}. The Kleene closure of B is the 
concatenation of an arbitrary number of strings, where each string is either 0 or 1. This is the set 
of all strings over the alphabet V = {0, 1}. That is, B* = V*. Finally, the Kleene closurebreak 
of C is the concatenation of the string 11 with itself an arbitrary number of times. Hence, C* is 
the set of strings consisting of an even number of 1s. That is, C* = {1% |n=0,1,2,...}. 4 


Finite-State Automata 


] We will now give a definition of a finite-state machine with no output. Such machines are also 
called finite-state automata, and that is the terminology we will use for them here. (Note: The 
singular of automata 1s automaton.) These machines differ from the finite-state machines studied 
in Section 13.2 in that they do not produce output, but they do have a set of final states. As we 
will see, they recognize strings that take the starting state to a final state. 


DEFINITION 3 


EXAMPLE 4 
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FIGURE 1 The State Diagram for a 
Finite-State Automaton. 


A finite-state automaton M = (S, I, f, so, F) consists of a finite set S of states, a finite input 
alphabet I, a transition function f that assigns a next state to every pair of state and input 
(so that f : S x I — S), an initial or start state s9, and a subset F' of S consisting of final 
(or accepting states). 


We can represent finite-state automata using either state tables or state diagrams. Final states 
are indicated in state diagrams by using double circles. 


Construct the state diagram for the finite-state automaton M = (S,/, f, so, F), where S = 
{so, $1, 52, 83}, J = {0, 1}, F = {so, 53}, and the transition function f is given in Table 1. 


Solution: The state diagram is shown in Figure 1. Note that because both the inputs 0 and 1 
take s2 to so, we write 0,1 over the edge from 5s to so. < 


EXTENDING THE TRANSITION FUNCTION The transition function f of a finite-state 
machine M = (S, /, f, 59, F) can be extended so that it is defined for all pairs of states and 
strings; that is, f can be extended to a function f : S x [* > S.Letx = x1x2...xx be a string 
in /*. Then f(s,, x) is the state obtained by using each successive symbol of x, from left to 
right, as input, starting with state s;. From s, we go on to state sy = f(s,, x;), then to state 
$3 = f(s2, x2), and so on, with f(s;,x) = f (sx, x). Formally, we can define this extended 
transition function f recursively for the deterministic finite-state machine M = (S, I, f, so, F) 
by 


(i) f(s,4) =s for every state s € S; and 
(ii) f(s, xa) = f(f(s, x), a) foralls € S,x € I*, andae/. 


STEPHEN COLE KLEENE (1909-1994) Stephen Kleene was born in Hartford, Connecticut. His mother, 
Alice Lena Cole, was a poet, and his father, Gustav Adolph Kleene, was an economics professor. Kleene attended 
Amherst College and received his Ph.D. from Princeton in 1934, where he studied under the famous logician 
Alonzo Church. Kleene joined the faculty of the University of Wisconsin in 1935, where he remained except for 
several leaves, including stays at the Institute for Advanced Study in Princeton. During World War II he was a 
navigation instructor at the Naval Reserve’s Midshipmen’s School and later served as the director of the Naval 
Research Laboratory. Kleene made significant contributions to the theory of recursive functions, investigating 
questions of computability and decidability, and proved one of the central results of automata theory. He served 
as the Acting Director of the Mathematics Research Center and as Dean of the College of Letters and Sciences 


at the University of Wisconsin. Kleene was a student of natural history. He discovered a previously undescribed variety of butterfly 
that is named after him. He was an avid hiker and climber. Kleene was also noted as a talented teller of anecdotes, using a powerful 
voice that could be heard several offices away. 
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We can use structural induction and this recursive definition to prove properties of this extended 
transition function. For example, in Exercise 15 we ask you to prove that 


f(s, xy) = f(f(s, x), y) 


for every state s € S and strings x € J* and y € I*. 


Language Recognition by Finite-State Machines 


Next, we define some terms that are used when studying the recognition by finite-state automata 
of certain sets of strings. 


DEFINITION 4 A string x is said to be recognized or accepted by the machine M = (S, I, f, so, F) if it takes 
the initial state sg to a final state, that is, f (so, x) is a state in F. The language recognized 
or accepted by the machine M, denoted by L(M), is the set of all strings that are recognized 
by M. Two finite-state automata are called equivalent if they recognize the same language. 


In Example 5 we will find the languages recognized by several finite-state automata. 


EXAMPLE 5 Determine the languages recognized by the finite-state automata M,, M2, and M3 in Figure 2. 


1 
Oi 1 
Start ()) 0 a 
M, 


M3 


FIGURE 2 Some Finite-State Automata. 
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Solution: The only final state of M, is so. The strings that take so to itself are those consisting 
of zero or more consecutive Is. Hence, L(M,) = {1" |n = 0,1, 2,...}. 

The only final state of M2 is sz. The only strings that take sp to s2 are | and OL. Hence, 
L(M2) = {1, O1}. 

The final states of M3 are so and s3. The only strings that take sq to itself are A, 0, O00, 000, .. . , 
that is, any string of zero or more consecutive Os. The only strings that take sg to s3 are a string 
of zero or more consecutive Os, followed by 10, followed by any string. Hence, L(M3) = 
{0", 0" 10x | =0,1,2,..., and x is any string}. 


DESIGNING FINITE-STATE AUTOMATA We can often construct a finite-state automaton 
that recognizes a given set of strings by carefully adding states and transitions and determining 
which of these states should be final states. When appropriate we include states that can keep 
track of some of the properties of the input string, providing the finite-state automaton with 
limited memory. Examples 6 and 7 illustrate some of the techniques that can be used to construct 
finite-state automata that recognize particular types of sets of strings. 


EXAMPLE 6 _ Construct deterministic finite-state automata that recognize each of these languages. 


Extra (a) the set of bit strings that begin with two Os 
Examples @ (b) the set of bit strings that contain two consecutive Os 
(c) the set of bit strings that do not contain two consecutive Os 
(d) the set of bit strings that end with two Os 
(e) the set of bit strings that contain at least two Os 


Solution: (a) Our goal is to construct a deterministic finite-state automaton that recog- 
nizes the set of bit strings that begin with two Os. Besides the start state so, we include a 
nonfinal state s;; we move to s; from sq if the first bit is a 0. Next, we add a final state s2, which 
we move to from s, if the second bit is a 0. When we have reached s2 we know that the first two 
input bits are both Os, so we stay in the state s2 no matter what the succeeding bits (if any) are. 
We move to a nonfinal state s3 from sq if the first bit is a 1 and from s; if the second bit is a 1. 
The reader should verify that the finite-state automaton in Figure 3(a) recognizes the set of bit 
strings that begin with two Os. 


FIGURE3 Deterministic Finite-State Automata Recognizing the Languages in Example 6. 
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EXAMPLE 7 


(b) Our goal is to construct a deterministic finite-state automaton that recognizes the set of 
bit strings that contain two consecutive Os. Besides the start state sg, we include a nonfinal 
state s;, which tells us that the last input bit seen is a 0, but either the bit before it was a 1, or 
this bit was the initial bit of the string. We include a final state sz that we move to from s; when 
the next input bit after a 0 is also a 0. If a 1 follows a 0 in the string (before we encounter two 
consecutive Os), we return to sg and begin looking for consecutive Os all over again. The reader 
should verify that the finite-state automaton in Figure 3(b) recognizes the set of bit strings that 
contain two consecutive Os. 


(c) Our goal is to construct a deterministic finite-state automaton that recognizes the set of 
bit strings that do not contain two consecutive Os. Besides the start state so, which should 
be a final state, we include a final state sj, which we move to from sg when 0 is the first 
input bit. When an input bit is a 1, we return to, or stay in, state sp. We add a state s2, which 
we move to from s; when the input bit is a 0. Reaching s2 tells us that we have seen two 
consecutive Os as input bits. We stay in state s2 once we have reached it; this state is not final. 
The reader should verify that the finite-state automaton in Figure 3(c) recognizes the set of bit 
strings that do not contain two consecutive Os. [The astute reader will notice the relationship 
between the finite-state automaton constructed here and the one constructed in part (b). See 
Exercise 39.] 


(d) Our goal is to construct a deterministic finite-state automaton that recognizes the set of bit 
strings that end with two Os. Besides the start state so, we include a nonfinal state s;, which 
we move to if the first bit is 0. We include a final state s2, which we move to from s, if the 
next input bit after a 0 is also a O. If an input of O follows a previous 0, we stay in state s2 
because the last two input bits are still Os. Once we are in state s2, an input bit of 1 sends us back 
to so, and we begin looking for consecutive Os all over again. We also return to so if the next 
input is a 1 when we are in state s;. The reader should verify that the finite-state automaton in 
Figure 3(d) recognizes the set of bit strings that end with two Os. 


(e) Our goal is to construct a deterministic finite-state automaton that recognizes the set of bit 
strings that contain two Os. Besides the start state, we include a state s;, which is not final; we 
stay in so until an input bit is a0 and we move to s; when we encounter the first 0 bit in the input. 
We add a final state s7, which we move to from s; once we encounter a second 0 bit. Whenever 
we encounter a | as input, we stay in the current state. Once we have reached s2, we remain 
there. Here, s; and sz are used to tell us that we have already seen one or two Os in the input 
string so far, respectively. The reader should verify that the finite-state automaton in Figure 3(e) 
recognizes the set of bit strings that contain two Os. 


Construct a deterministic finite-state automaton that recognizes the set of bit strings that contain 
an odd number of Is and that end with at least two consecutive Os. 


Solution: We can build a deterministic finite-state automaton that recognizes the specified set 
by including states that keep track of both the parity of the number of | bits and whether we 
have seen no, one, or at least two Os at the end of the input string. 

The start state sg can be used to tell us that the input read so far contains an even number 
of 1s and ends with no Os (that is, is empty or ends with a 1). Besides the start state, we include 
five more states. We move to states 51, s2, 53, 54, and s5, respectively, when the input string read 
so far contains an even number of 1s and ends with one 0; when it contains an even number 
of 1s and ends with at least two Os; when it contains an odd number of Is and ends with no Os; 
when it contains an odd number of Is and ends with one 0; and when it contains an odd number 
of 1s and ends with two Os. The state ss is a final state. 

The reader should verify that the finite-state automaton in Figure 4 recognizes the set of bit 
strings that contain an odd number of Is and end with at least two consecutive Os. < 
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Start 
Number of even even even odd odd odd 
Is 
Number of 0 1 22 0 1 22 
Os at end 


FIGURE4 A Deterministic Finite-State Automaton Recognizing the Set of Bit Strings Containing an Odd 
Number of 1s and Ending with at Least Two 0s. 


EXAMPLE 8 


EQUIVALENT FINITE-STATE AUTOMATA In Definition 4 we specified that two finite- 
state automata are equivalent if they recognize the same language. Example 8 provides an 
example of two equivalent deterministic finite-state machines. 


Show that the two finite-state automata Mp and M, shown in Figure 5 are equivalent. 


Solution: For a string x to be recognized by Mo, x must take us from sq to the final state s1 or 
the final state s4. The only string that takes us from so to s; 1s the string |. The strings that take 
us from so to s4 are those strings that begin with a 0, which takes us from so to 52, followed by 
zero or more additional Os, which keep the machine in state 52, followed by a 1, which takes us 
from state s2 to the final state s4. All other strings take us from so to a state that is not final. (We 
leave it to the reader to fill in the details.) We conclude that L(Mo) is the set of strings of zero 
or more 0 bits followed by a final 1. 

For a string x to be recognized by M1, x must take us from so to the final state 51. So, for x 
to be recognized, it must begin with some number of Os, which leave us in state so, followed by 


0 
Start ‘ (=) 0.1 
1 0,1 
M, 


FIGURE 5 Mo and M, Are Equivalent Finite-State Automata. 


872 


13 / Modeling Computation 


a 1, which takes us to the final state s). A string of all zeros is not recognized because it leaves us 
in state so, which is not final. All strings that contain a 0 after 1 are not recognized because they 
take us to state s2, which is not final. It follows that L(M,) is the same as L(Mo). We conclude 
that Mo and M, are equivalent. 

Note that the finite-state machine M, only has three states. No finite state machine with 
fewer than three states can be used to recognize the set of all strings of zero or more 0 bits 
followed by a 1| (see Exercise 37). < 


As Example 8 shows, a finite-state automaton may have more states than one equivalent 
to it. In fact, algorithms used to construct finite-state automata to recognize certain languages 
may have many more states than necessary. Using unnecessarily large finite-state machines to 
recognize languages can make both hardware and software applications inefficient and costly. 
This problem arises when finite-state automata are used in compilers, which translate computer 
programs to a language a computer can understand (object code). 

Exercises 58-61 develop a procedure that constructs a finite-state automaton with the fewest 
states possible among all finite-state automata equivalent to a given finite-state automaton. This 
procedure is known as machine minimization. The minimization procedure described in these 
exercises reduces the number of states by replacing states with equivalence classes of states with 
respect to an equivalence relation in which two states are equivalent if every input string either 
sends both states to a final state or sends both to a state that is not final. Before the minimization 
procedure begins, all states that cannot be reached from the start state using any input string are 
first removed; removing these does not change the language recognized. 


GRACE BREWSTER MURRAY HOPPER (1906-1992) Grace Hopper, born in New York City, displayed an 
intense curiosity as a child with how things worked. At the age of seven, she disassembled alarm clocks to discover 
their mechanisms. She inherited her love of mathematics from her mother, who received special permission to 
study geometry (but not algebra and trigonometry) at a time when women were actively discouraged from such 
study. Hopper was inspired by her father, a successful insurance broker, who had lost his legs from circulatory 
problems. He told his children they could do anything if they put their minds to it. He inspired Hopper to pursue 
higher education and not conform to the usual roles for females. Her parents made sure that she had an excellent 
education; she attended private schools for girls in New York. Hopper entered Vassar College in 1924, where she 
majored in mathematics and physics; she graduated in 1928. She received a masters degree in mathematics from 
Yale University in 1930. In 1930 she also married an English instructor at the New York School of Commerce; she later divorced and 
did not have children. Hopper was a mathematics professor at Vassar from 1931 until 1943, earning a Ph.D. from Yale in 1934. 

After the attack on Pearl Harbor, Hopper, coming from a family with strong military traditions, decided to leave her academic 
position and join the Navy WAVES. To enlist, she needed special permission to leave her strategic position as a mathematics professor, 
as well as a waiver for weighing too little. In December 1943, she was sworn into the Navy Reserve and trained at the Midshipman’s 
School for Women. Hopper was assigned to work at the Naval Ordnance Laboratory] at Harvard University. She wrote programs for 
the world’s first large-scale automatically sequenced digital computer, which was used to help aim Navy artillery in varying weather. 
Hopper has been credited with coining the term “bug” to refer to a hardware glitch, but it was used at Harvard prior to her arrival 
there. However, it is true that Hopper and her programming team found a moth in one of the relays in the computer hardware that 
shut the system down. This famous moth was pasted into a lab book. In the 1950s Hopper coined the term “debug” for the process 
of removing programming errors. 

In 1946, when the Navy told her that she was too old for active service, Hopper chose to remain at Harvard as a civilian research 
fellow. In 1949 she left Harvard to join the Eckert-Mauchly Computer Corporation, where she helped develop the first commercial 
computer, UNIVAC. Hopper remained with this company when it was taken over by Remington Rand and when Remington Rand 
merged with the Sperry Corporation. She was a visionary for the potential power of computers; she understood that computers would 
become widely used if tools that were both programmer-friendly and application-friendly could be developed. In particular, she 
believed that computer programs could be written in English, rather than using machine instructions. To help achieve this goal, she 
developed the first compiler. She published the first research paper on compilers in 1952. Hopper is also known as the mother of the 
computer language COBOL; members of Hopper’s staff helped to frame the basic language design for COBOL using their earlier 
work as a basis. 

In 1966, Hopper retired from the Navy Reserve. However, only seven months later, the Navy recalled her from retirement to 
help standardize high-level naval computer languages. In 1983 she was promoted to the rank of Commodore by special Presidential 
appointment, and in 1985 she was elevated to the rank of Rear Admiral. Her retirement from the Navy, at the age of 80, was held on 
the USS Constitution. 
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EXAMPLE 9 


EXAMPLE 10 
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Nondeterministic Finite-State Automata 


The finite-state automata discussed so far are deterministic, because for each pair of state and 
input value there is a unique next state given by the transition function. There is another important 
type of finite-state automaton in which there may be several possible next states for each pair 
of input value and state. Such machines are called nondeterministic. Nondeterministic finite- 
state automata are important in determining which languages can be recognized by a finite-state 
automaton. 


A nondeterministic finite-state automaton M = (S, 1, f, so, F) consists of a set S of states, 
an input alphabet /, a transition function f that assigns a set of states to each pair of state and 
input (so that f : S x I + P(S)), a starting state so, and a subset F of S consisting of the 
final states. 


We can represent nondeterministic finite-state automata using state tables or state diagrams. 
When we use a state table, for each pair of state and input value we give a list of possible 
next states. In the state diagram, we include an edge from each state to all possible next states, 
labeling edges with the input or inputs that lead to this transition. 


Find the state diagram for the nondeterministic finite-state automaton with the state table shown 
in Table 2. The final states are s2 and s3. 


Solution: The state diagram for this automaton is shown in Figure 6. < 


Find the state table for the nondeterministic finite-state automaton with the state diagram shown 
in Figure 7. 


Solution: The state table is given as Table 3. < 


What does it mean for a nondeterministic finite-state automaton to recognize a string 
xX = xX1xX2...x,? The first input symbol x; takes the starting state sp to a set S, of states. The 
next input symbol x2 takes each of the states in S, to a set of states. Let Sz be the union of these 
sets. We continue this process, including at a stage all states obtained using a state obtained 
at the previous stage and the current input symbol. We recognize, or accept, the string x if 
there is a final state in the set of all states that can be obtained from so using x. The language 
recognized by a nondeterministic finite-state automaton is the set of all strings recognized by 
this automaton. 


TABLE 2 
f 


FIGURE 6 The Nondeterministic 
Finite-State Automaton with State 


saa Table Given in Table 2. 
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EXAMPLE 11 


THEOREM 1 


? 


EXAMPLE 12 


TABLE 3 


lee 3 
State 


Start 


O) an 
FIGURE 7 A Nondeterministic Finite-State ' 
Automaton. 


Find the language recognized by the nondeterministic finite-state automaton shown in Figure 7. 


Solution: Because sq is a final state, and there is a transition from so to itself when 0 is the input, 
the machine recognizes all strings consisting of zero or more consecutive Os. Furthermore, 
because sq is a final state, any string that has sq in the set of states that can be reached from sg 
with this input string is recognized. The only such strings are strings consisting of zero or more 
consecutive Os followed by 01 or 11. Because sp and sq are the only final states, the language 
recognized by the machine is {0”, 0”01, 0"11 | n > O}. 


One important fact is that a language recognized by a nondeterministic finite-state automaton 
is also recognized by a deterministic finite-state automaton. We will take advantage of this fact in 
Section 13.4 when we will determine which languages are recognized by finite-state automata. 


If the language L is recognized by a nondeterministic finite-state automaton Mo, then L is 
also recognized by a deterministic finite-state automaton M. 


Proof: We will describe how to construct the deterministic finite-state automaton M that rec- 
ognizes L from Mo, the nondeterministic finite-state automaton that recognizes this language. 
Each state in M, will be made up of a set of states in Mo. The start symbol of M is {so}, which 
is the set containing the start state of Mo. The input set of M, is the same as the input set of Mo. 

Given a state {5;,, 8i,,...,5i,} of M1, the input symbol x takes this state to the union 
of the sets of next states for the elements of this set, that is, the union of the sets f(s;,, x), 
Sf (Sin, x), -.., f (Siz, x). The states of Mj are all the subsets of S, the set of states of Mo, that are 
obtained in this way starting with sq. (There are as many as 2” states in the deterministic machine, 
where n is the number of states in the nondeterministic machine, because all subsets may occur 
as states, including the empty set, although usually far fewer states occur.) The final states 
of M, are those sets that contain a final state of Mo. 

Suppose that an input string is recognized by Mo. Then one of the states that can be reached 
from sg using this input string is a final state (the reader should provide an inductive proof of 
this). This means that in M, this input string leads from {5} to a set of states of Mo that contains 
a final state. This subset is a final state of M1, so this string is also recognized by M1. Also, an 
input string not recognized by Mo does not lead to any final states in Mo. (The reader should 
provide the details that prove this statement.) Consequently, this input string does not lead from 
{so} to a final state in M). | 


Find a deterministic finite-state automaton that recognizes the same language as the nondeter- 
ministic finite-state automaton in Example 10. 
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FIGURE 8 A Deterministic Automaton Equivalent to the 
Nondeterministic Automaton in Example 10. 


Solution: The deterministic automaton shown in Figure 8 is constructed from the nonde- 
terministic automaton in Example 10. The states of this deterministic automaton are sub- 
sets of the set of all states of the nondeterministic machine. The next state of a subset un- 
der an input symbol is the subset containing the next states in the nondeterministic ma- 
chine of all elements in this subset. For instance, on input of 0, {so} goes to {so, s2}, 
because sg has transitions to itself and to sz in the nondeterministic machine; the set {so, s2} 
goes to {s,, 54} on input of 1, because so goes just to s; and s2 goes just to s4 on input of | in 
the nondeterministic machine; and the set {s;, 54} goes to {s3} on input of 0, because s; and s4 
both go to just s3 on input of 0 in the deterministic machine. All subsets that are obtained in 
this way are included in the deterministic finite-state machine. Note that the empty set is one of 
the states of this machine, because it is the subset containing all the next states of {s3} on input 
of 1. The start state is {sq}, and the set of final states are all those that include so or 54. 


Exercises 
1. Let A = {0,11} and B = {00, 01}. Find each of these c) {11} {0}*{01} d) {11}*{01}* 
sets. e) {111}*{0}*{1} f) {11, 0} {00, 101} 
a) AB c) A? d) B? 10. Determine whether the string 01001 is in each of these 
2. Show that if A is a set of strings, then AY = GA = 0. sets. 
3. Find all pairs of sets of strings A and B for which a) {0, 1}* b) {0}*{10}{1}* 
AB = {10, 111, 1010, 1000, 10111, 101000}. c) {010}*{0}*{1} d) {010, 011} {00, 01} 
2k 2k 2k 
4. Show that these equalities hold. ©) {00} {0}"{01} f) {O1}" {01} 
x 11. Determine whether each of these strings is recognized by 
ep ag the deterministic finite-state automaton in Figure 1 
b) (A*)* = A* for every set of strings A 8 : 


5. Describe the elements of the set A* for these values of A. 
b) {111} c) {0,01} d) {1, 101} 12. Determine whether each of these strings is recognized by 


a) {10} 


6. Let V be an alphabet, and let A and B be subsets of V*. 
Show that |A B| < |A||B]. 
7. Let V be an alphabet, and let A and B be subsets of V* 


a) 111 b) 0011 c) 1010111 d) 011011011 


the deterministic finite-state automaton in Figure 1. 
a) 010 b) 1101 ce) 1111110 d) 010101010 
13. Determine whether all the strings in each of these sets are 


with A C B. Show that A* C B* recognized by the deterministic finite-state automaton in 
on es Figure 1. 
8. Suppose that A is a subset of V*, where V is an alphabet. 
ee . a) {0}* b) {0} {0"" ——&) {1} {0}* 
Prove or disprove each of these statements. - 
, ; 2 d) {01}* e) {0}*{1} f) {1} {0, 1}* 
a) ACA b) if A= A*,thenae A : . ae ; 
©) AQ) =A d) (A*)* = A* 14. Show thatif M = (S, /, f, so, F) isadeterministic finite- 
e) AtA= AM f) |A"| =|Al" state automaton and f(s, x) = s for the state s € § and 
a j ao the input string x € 7*, then f(s, x”) = s for every non- 
9. Determine whether the string 11101 is in each of these negative integer n. (Here x” is the concatenation of n 
Bets copies of the string x, defined recursively in Exercise 37 
a) {0, 1}* b) {1}*{0}*(1}* in Section 5.3.) 
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15. Given a deterministic finite-state automaton M = 
(S, 7, f, So, F), use structural induction and the recursive 
definition of the extended transition function f to prove 
that f(s, xy) = f(f(s, x), y) for all states 5 € S and all 
strings x € /* andy € I*. 

In Exercises 16—22 find the language recognized by the given 

deterministic finite-state automaton. 


16. 


23. 


24, 


25. 


26 


27 


28 


29 


30 


31 


32. 


33. 


34 


35. 


36. 


37. 


38. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings beginning with 01. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that end with 10. 


Construct a deterministic finite-state automaton that 
recognizes the set of all bit strings that contain the 
string 101. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that do not contain three 
consecutive Os. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain exactly 
three Os. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain at least 
three Os. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain three consec- 
utive 1s. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that begin with 0 or 
with 11. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that begin and end with 11. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain an even num- 
ber of Is. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain an odd number 
of Os. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain an even num- 
ber of Os and an odd number of 1s. 


Construct a finite-state automaton that recognizes the set 
of bit strings consisting of a 0 followed by a string with 
an odd number of Is. 


Construct a finite-state automaton with four states that 
recognizes the set of bit strings containing an even num- 
ber of 1s and an odd number of Os. 


Show that there is no finite-state automaton with two 
states that recognizes the set of all bit strings that have 
one or more | bits and end with a 0. 


Show that there is no finite-state automaton with three 
states that recognizes the set of bit strings containing an 
even number of 1s and an even number of Os. 


39, Explain how you can change the deterministic finite-state 
automaton M so that the changed automaton recognizes 
the set /* — L(M). 

40. Use Exercise 39 and finite-state automata constructed in 

Example 6 to find deterministic finite-state automata that 

recognize each of these sets. 


a) the set of bit strings that do not begin with two Os 


b) the set of bit strings that do not end with two Os 


c) the set of bit strings that contain at most one 0 (that 
is, that do not contain at least two Os) 


41. Use the procedure you described in Exercise 39 and the 
finite-state automata you constructed in Exercise 25 to 
find a deterministic finite-state automaton that recognizes 
the set of all bit strings that do not contain the string 101. 


42. Use the procedure you described in Exercise 39 and the 
finite-state automaton you constructed in Exercise 29 to 
find a deterministic finite-state automaton that recognizes 
the set of all bit strings that do not contain three consec- 
utive Is. 


In Exercises 43-49 find the language recognized by the given 
nondeterministic finite-state automaton. 


43. Start (= () 1 


47. 


Start 
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1,0 


50. Find a deterministic finite-state automaton that recog- 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 43. 


51. Find a deterministic finite-state automaton that recog- 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 44. 

52. Find a deterministic finite-state automaton that recog- 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 45. 

53. Find a deterministic finite-state automaton that recog- 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 46. 

54. Find a deterministic finite-state automaton that recog- 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 47. 

55. Find a deterministic finite-state automaton that recog- 
nizes each of these sets. 

a) {0} b) {1, 00} 
c) {17 |n=2,3,4,...} 

56. Find a nondeterministic finite-state automaton that recog- 
nizes each of the languages in Exercise 55, and has fewer 
states, if possible, than the deterministic automaton you 
found in that exercise. 


*57. Show that there is no finite-state automaton that recog- 


nizes the set of bit strings containing an equal number 
of Os and Is. 


In Exercises 58-62 we introduce a technique for construct- 
ing a deterministic finite-state machine equivalent to a given 
deterministic finite-state machine with the least number of 
states possible. Suppose that M = (S, J, f, 50, F) is a finite- 
state automaton and that k is a nonnegative integer. Let Rx be 
the relation on the set S$ of states of M such that sRxt if and 
only if for every input string x with /(x) < k [where /(x) is the 
length of x, as usual], f(s, x) and f(t, x) are both final states 
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or both not final states. Furthermore, let R,, be the relation on 
the set of states of M such that sR,t if and only if for every 
input string x, regardless of length, f(s,x) and f(t, x) are 
both final states or both not final states. 

*58. a) Show that for every nonnegative integer k, Ry is an 


b) 


c) 


equivalence relation on S. We say that two states s 
and t are k-equivalent if s Rx. 

Show that R,. is an equivalence relation on S. We say 
that two states s and ¢ are +-equivalent if s R,t. 
Show that if s and ¢ are two k-equivalent states of M, 
where k is a positive integer, then s and k are also 


*60. a) Show that s and t¢ are 0-equivalent if and only if either 


b) 


c) 


both s and ¢ are final states or neither s nor ¢ is a final 
state. Conclude that each final state of M, which is an 
R,-equivalence class, contains only final states of M. 
Show that if k is a positive integer, then s and ¢ are k- 
equivalent if and only if s and ¢ are (k — 1)-equivalent 
and for every input symbola € J, f(s, a) and f(t, a) 
are (k — 1)-equivalent. Conclude that the transition 
function f is well-defined. 

Describe a procedure that can be used to construct the 
quotient automaton of a finite-automaton M. 


(k — 1)-equivalent **61. a) Show that if M is a finite-state automaton, then the 
d) Show that the equivalence classes of R; are a refine- quotient automaton M recognizes the same language 

ment of the equivalence classes of Rx_, if k is a pos- as M. 

itive integer. (The refinement of a partition of a set is b) Show that if M is a finite-state automaton with the 

defined in the preamble to Exercise 49 in Section 9.5.) property that for every state s of M there is a string 
e) Show that if s and ¢ are k-equivalent for every non- x € I* such that f(so, x) = s, then the quotient au- 

negative integer k, then they are +-equivalent. tomaton M has the minimum number of states of any 
f) Show that all states in a given R,-equivalence class finite-state automaton equivalent to M. 

are final states or all are not final states. 62. Answer these questions about the finite-state automaton 


g) Show that if s and ¢t are R,-equivalent, then f(s, a) 
and f(t, a) are also R,-equivalent for alla € I. 


Show that there is a nonnegative integer n such that the set 
of n-equivalence classes of states of M is the same as the 
set of (n + 1)-equivalence classes of states of M. Then 
show for this integer n, the set of n-equivalence classes 
of states of M equals the set of «-equivalence classes of 
states of M. 

The quotient automaton M of the deterministic finite-state 
automaton M = (S,/, f, so, F) is the finite-state automaton 
(S, 1, f. lsolrx, F), where the set of states S is the set of 
*-equivalence classes of S, the transition function f is de- 
fined by f([s]r,,@) =L f(s, a) |r, for all states [s]p, of M 
and input symbols a € J, and F is the set consisting of R,- 
equivalence classes of final states of M. 
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M shown here. 
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a) Find the k-equivalence classes of M for k = 0, 1, 2, 
and 3. Also, find the *-equivalence classes of M. 
b) Construct the quotient automaton M of M. 


We have seen that finite-state automata can be used as language recognizers. What sets can be 
recognized by these machines? Although this seems like an extremely difficult problem, there 
is a simple characterization of the sets that can be recognized by finite state automata. This 
problem was first solved in 1956 by the American mathematician Stephen Kleene. He showed 
that there is a finite-state automaton that recognizes a set if and only if this set can be built 
up from the null set, the empty string, and singleton strings by taking concatenations, unions, 
and Kleene closures, in arbitrary order. Sets that can be built up in this way are called regular 
sets. Regular grammars were defined in Section 13.1. Because of the terminology used, it is 
not surprising that there is a connection between regular sets, which are the sets recognized 
by finite-state automata, and regular grammars. In particular, a set is regular if and only if it is 
generated by a regular grammar. 

Finally, there are sets that cannot be recognized by any finite-state automata. We will give 
an example of such a set. We will briefly discuss more powerful models of computation, such as 
pushdown automata and Turing machines, at the end of this section. The regular sets are those 


DEFINITION 1 


EXAMPLE 1 


EXAMPLE 2 
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that can be formed using the operations of concatenation, union, and Kleene closure in arbitrary 
order, starting with the empty set, the set consisting of the empty string, and singleton sets. We 
will see that the regular sets are those that can be recognized using a finite-state automaton. To 
define regular sets we first need to define regular expressions. 


The regular expressions over a set I are defined recursively by: 


the symbol @ is a regular expression; 

the symbol A is a regular expression; 

the symbol x is a regular expression whenever x € /; 

the symbols (AB), (A U B), and A®* are regular expressions whenever A 
and B are regular expressions. 


Each regular expression represents a set specified by these rules: 


% represents the empty set, that is, the set with no strings; 

A represents the set {A}, which is the set containing the empty string; 

x represents the set {x} containing the string with one symbol x; 

(AB) represents the concatenation of the sets represented by A and by B; 
(A U B) represents the union of the sets represented by A and by B; 

A* represents the Kleene closure of the set represented by A. 


Sets represented by regular expressions are called regular sets. Henceforth regular expressions 
will be used to describe regular sets, so when we refer to the regular set A, we will mean the 
regular set represented by the regular expression A. Note that we will leave out outer parentheses 
from regular expressions when they are not needed. 

Example | shows how regular expressions are used to specify regular sets. 


What are the strings in the regular sets specified by the regular expressions 10*, (10)*, 0 U 01, 
0(0 U 1)*, and (0*1)*? 


Solution: The regular sets represented by these expressions are given in Table 1, as the reader 
should verify. < 


Finding a regular expression that specifies a given set can be quite tricky, as Example 2 
illustrates. 


Find a regular expression that specifies each of these sets: 


(a) the set of bit strings with even length 
(b) the set of bit strings ending with a 0 and not containing 11 
(c) the set of bit strings containing an odd number of Os 


TABLE 1 
Expression Strings 


10* a 1 followed by any number of Os (including no zeros) 


(10)* any number of copies of 10 (including the null string) 
0U01 the string 0 or the string 01 
0(0 U 1)* any string beginning with 0 


(0*1)* any string not ending with 0 
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Solution: (a) To construct a regular expression for the set of bit strings with even length, we 
use the fact that such a string can be obtained by concatenating zero or more strings each 
consisting of two bits. The set of strings of two bits is specified by the regular expression 
(00 U01U10U 11). Consequently, the set of strings with even length is specified by 
(00 U01 U10U 11)*. 


(b) A bit string ending with a 0 and not containing 11 must be the concatenation of one or 
more strings where each string is either a 0 or a 10. (To see this, note that such a bit string must 
consist of 0 bits or 1 bits each followed by a 0; the string cannot end with a single 1 because 
we know it ends with a 0.) It follows that the regular expression (0 U 10)* (0 U 10) specifies the 
set of bit strings that do not contain 11 and end with a 0. [Note that the set specified by (0 U 
10)* includes the empty string, which is not in this set, because the empty string does not end 
with a 0.] 


(c) A bit string containing an odd number of Os must contain at least one 0, which tells us 
that it starts with zero or more Is, followed by a 0, followed by zero or more Is. That is, each 
such bit string begins with a string of the form 1/01* for nonnegative integers j and k. Because 
the bit string contains an odd number of Os, additional bits after this initial block can be split into 
blocks each starting with a 0 and containing one more 0. Each such block is of the form 017014, 
where p and qg are nonnegative integers. Consequently, the regular expression 1*01*(01*01*)* 
specifies the set of bit strings with an odd number of Os. 


Kleene’s Theorem 


In 1956 Kleene proved that regular sets are the sets that are recognized by a finite-state automaton. 
Consequently, this important result is called Kleene’s theorem. 


KLEENE’S THEOREM A set is regular if and only if it is recognized by a finite-state 
automaton. 


Kleene’s theorem is one of the central results in automata theory. We will prove the only if part of 
this theorem, namely, that every regular set is recognized by a finite-state automaton. The proof 
of the if part, that a set recognized by a finite-state automaton is regular, is left as an exercise 
for the reader. 


Proof: Recall that a regular set is defined in terms of regular expressions, which are defined 
recursively. We can prove that every regular set is recognized by a finite-state automaton if we 
can do the following things. 


. Show that J is recognized by a finite-state automaton. 

. Show that {A} is recognized by a finite-state automaton. 

. Show that {a} is recognized by a finite-state automaton whenever a is a symbol in /. 
. Show that AB is recognized by a finite-state automaton whenever both A and B are. 


nA WN Re 


. Show that A U B is recognized by a finite-state automaton whenever both A and B are. 
6. Show that A* is recognized by a finite-state automaton whenever A is. 


We now consider each of these tasks. First, we show that Y is recognized by a nondeterministic 
finite-state automaton. To do this, all we need is an automaton with no final states. Such an 
automaton is shown in Figure I (a). 
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sn +(e) son —+(@)) Start -(*) as 
(b) (c) 


(a) 


FIGURE 1 Nondeterministic Finite-State Automata That Recognize Some Basic Sets. 


Second, we show that {A} is recognized by a finite-state automaton. To do this, all we need 
is an automaton that recognizes i, the null string, but not any other string. This can be done by 
making the start state so a final state and having no transitions, so that no other string takes sg 
to a final state. The nondeterministic automaton in Figure 1(b) shows such a machine. 

Third, we show that {a} is recognized by a nondeterministic finite-state automaton. To do 
this, we can use a machine with a starting state sg and a final state s;. We have a transition from 
Sg to s; when the input is a, and no other transitions. The only string recognized by this machine 
is a. This machine is shown in Figure 1(c). 

Next, we show that AB and A U B can be recognized by finite-state automata if A and B 
are languages recognized by finite-state automata. Suppose that A is recognized by My = 
(Sa, Ufa, sa, Fa) and B is recognized by Mg = (Sz, I fp. 5B, Fp). 

We begin by constructing a finite-state machine Mag = (Sap, /, fap, SAB, Fag) that recog- 
nizes AB, the concatenation of A and B. We build such a machine by combining the machines 
for A and B in series, so a string in A takes the combined machine from s4, the start state 
of M4, to sg, the start state of Mg. A string in B should take the combined machine from sz to 
a final state of the combined machine. Consequently, we make the following construction. 
Let Sap be S4 U Sg. [Note that we can assume that S,4 and Sg are disjoint.] The start- 
ing state sap is the same as s4. The set of final states, F4g, is the set of final states of 
Mg with sap included if and only if A € AN B. The transitions in M,p include all tran- 
sitions in M, and in Mg, as well as some new transitions. For every transition in M, 
that leads to a final state, we form a transition in M,g from the same state to sg, on the 
same input. In this way, a string in A takes Map from s4g to sg, and then a string in B 
takes sg to a final state of M4g. Moreover, for every transition from sg we form a transition 
in Mag from s,4z to the same state. Figure 2(a) contains an illustration of this construction. 

We now construct a machine Maug = (Saus,/, faus, SAuB, FAup) that recognizes 
AUB. This automaton can be constructed by combining My, and Mz in parallel, using a 
new Start state that has the transitions that both s4 and sg have. Let Saug = S4 U Sp U {sup}, 
where sug 1s a new State that is the start state of Mug. Let the set of final states F4ug be 
Fa U Fp U {saup} if A € AU B, and Fy, U Fz otherwise. The transitions in Mug include all 
those in M, and in Mg. Also, for each transition from s, to a state s on input i we include a 
transition from s4uyg to s on input 7, and for each transition from sz to a state s on input i we 
include a transition from s4ug to s on input 7. In this way, a string in A leads from s4ug to 
a final state in the new machine, and a string in B leads from sug to a final state in the new 
machine. Figure 2(b) illustrates the construction of Mus. 

Finally, we construct M,* = (S4*, 1, fy*,8,4*, /4*), a machine that recognizes A*, the 
Kleene closure of A. Let $ At include all states in S,4 and one additional state s A* which is 
the starting state for the new machine. The set of final states F,« includes all states in F4 
as well as the start state s,*, because 4 must be recognized. To recognize concatenations of 
arbitrarily many strings from A, we include all the transitions in M4, as well as transitions 
from s Ae that match the transitions from s4, and transitions from each final state that match the 
transitions from s,4. With this set of transitions, a string made up of concatenations of strings 
from A will take s,* to a final state when the first string in A has been read, returning to a 
final state when the second string in A has been read, and so on. Figure 2(c) illustrates the 
construction we used. 4 


A nondeterministic finite-state automaton can be constructed for any regular set using the 
procedure described in this proof. We illustrate how this is done with Example 3. 
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a Transition to final state in M, produces a transition to sp. 
‘AP B 


Transition from sz in Mg, produces a transition from s4p = S4. 


Start state is 84, = S4, which is final if s, and sz are final. Final states include all final states of Mp. 


(b) 


Start 


Maus 


C) ie -O) 


Final states are the final states in M, and Mg. 


Sup 1S the new start state, which is final if s4 or sz is final. 


(c) Transitions from s, produce A transitions from s, + and all final states of My. 


4» is the new start state, which is a final state. Final states include all final states in M,. 


FIGURE 2 Building Automata to Recognize Concatenations, Unions, and Kleene Closures. 


EXAMPLE 3 


Construct a nondeterministic finite-state automaton that recognizes the regular set 1* U 01. 


Solution: We begin by building a machine that recognizes 1*. This is done using the machine 
that recognizes 1 and then using the construction for M ,* described in the proof. Next, we 
build a machine that recognizes 01, using machines that recognize 0 and 1 and the construction 
in the proof for M4 z. Finally, using the construction in the proof for M4ug, we construct the 
machine for 1* U 01. The finite-state automata used in this construction are shown in Figure 3. 
The states in the successive machines have been labeled using different subscripts, even when a 
state is formed from one previously used in another machine. Note that the construction given 
here does not produce the simplest machine that recognizes 1* U01. A much simpler machine 
that recognizes this set is shown in Figure 3(b). 


Regular Sets and Regular Grammars 


In Section 13.1 we introduced phrase-structure grammars and defined different types of gram- 
mars. In particular we defined regular, or type 3, grammars, which are grammars of the 
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FIGURE 3  Nondeterministic Finite-State Automata Recognizing 1* U 01. 


form G = (V,T,S, P), where each production is of the form S$ > 4, A> a, or A> aB, 
where a is a terminal symbol, and A and B are nonterminal symbols. As the terminology 
suggests, there is a close connection between regular grammars and regular sets. 


THEOREM 2 A set is generated by a regular grammar if and only if it is a regular set. 
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FIGURE4 A 
Nondeterministic 
Finite-State 
Automaton 
Recognizing L(G). 


EXAMPLE 4 


FIGURE 5 
A Finite-State 
Automaton. 


EXAMPLE 5 


Proof: First we show that a set generated by a regular grammar is a regular set. Suppose 
thatG = (V,T, S, P)isaregular grammar generating the set L(G). To show that L(G) is regular 
we will build a nondeterministic finite-state machine M = (S, I, f, so, F) thatrecognizes L(G). 
Let S, the set of states, contain a state s4 for each nonterminal symbol A of G and an additional 
state sr, which is a final state. The start state sp is the state formed from the start symbol S. 
The transitions of M are formed from the productions of G in the following way. A transition 
from s4 to sf on input of a is included if A — a is a production, and a transition from s 4 to sg 
on input of a is included if A + aB is a production. The set of final states includes sr and also 
includes so if S — A is a production in G. It is not hard to show that the language recognized 
by M equals the language generated by the grammar G, that is, L(M) = L(G). This can be 
done by determining the words that lead to a final state. The details are left as an exercise for 
the reader. | 


Before giving the proof of the converse, we illustrate how a nondeterministic machine is 
constructed that recognizes the same set as a regular grammar. 


Construct a nondeterministic finite-state automaton that recognizes the language generated by 
the regular grammar G = (V, T, S, P), where V = {0, 1, A, S}, T = {0, 1}, and the productions 
in PareS > 1A,S > 0,S >A, A— 0A, A — 1A, and A — 1. 


Solution: The state diagram for a nondeterministic finite-state automaton that recognizes L(G) 
is shown in Figure 4. This automaton is constructed following the procedure described in the 
proof. In this automaton, sg is the state corresponding to S, s1 is the state corresponding to A, 
and sp is the final state. 


We now complete the proof of Theorem 2. 


Proof: We now show that if a set is regular, then there is a regular grammar that generates 
it. Suppose that M is a finite-state machine that recognizes this set with the property that so, 
the starting state of M, is never the next state for a transition. (We can find such a machine 
by Exercise 20.) The grammar G = (V, 7, S, P) is defined as follows. The set V of symbols 
of G is formed by assigning a symbol to each state of S and each input symbol in /. The 
set T of terminal symbols of G is the set J. The start symbol S is the symbol formed 
from the start state so. The set P of productions in G is formed from the transitions in 
M. In particular, if the state s goes to a final state under input a, then the production 
As — a 1s included in P, where A, is the nonterminal symbol formed from the state s. 
If the state s goes to the state ¢ on input a, then the production As; — aA; is included 
in P. The production S$ — 4 is included in P if and only if A ¢ L(M). Because the productions 
of G correspond to the transitions of M and the productions leading to terminals correspond to 
transitions to final states, it is not hard to show that L(G) = L(M). We leave the details as an 
exercise for the reader. | 


Example 5 illustrates the construction used to produce a grammar from an automaton that 
generates the language recognized by this automaton. 


Find a regular grammar that generates the regular set recognized by the finite-state automaton 
shown in Figure 5. 


Solution: The grammar G = (V, T, S, P) generates the set recognized by this automaton where 
V ={S, A, B,0, 1}, the symbols S$, A, and B correspond to the states so, 51, and s2, respec- 
tively, T = {0, 1}, S is the start symbol; and the productions are $S > 0A, S—> 1B, S—> 1, 
S—>i,A—> 0A,A> 1B,A—> 1, B > OA, B > 1B,and B > 1. < 


EXAMPLE 6 


? 


Links 
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FIGURE 6 The Path Produced by 0" 1%. 


A Set Not Recognized by a Finite-State Automaton 


We have seen that a set is recognized by a finite state automaton if and only if it is regular. We 
will now show that there are sets that are not regular by describing one such set. The technique 
used to show that this set is not regular illustrates an important method for showing that certain 
sets are not regular. 


Show that the set {0"1” | = 0, 1,2,...}, made up of all strings consisting of a block of Os 
followed by a block of an equal number of Is, is not regular. 


Solution: Suppose that this set were regular. Then there would be a nondeterministic finite-state 
automaton M = (S, /, f, so, F) recognizing it. Let N be the number of states in this machine, 
that is, N = |S|. Because M recognizes all strings made up of a number of Os followed by an 
equal number of 1s, M must recognize 0' 1%. Let so, 51, 52, ..., 82 be the sequence of states 
that is obtained starting at so and using the symbols of 0” 1% as input, so that s; = f (so, 0), 
82 = f(81,0),...,5n = f(sw-1, 9), svgi = f(sn, 1),..., 52n = f(S2n-1, 1). Note that s2y 
is a final state. 

Because there are only WN states, the pigeonhole principle shows that at least two of the 
first N + 1 of the states, which are so, ..., 5), must be the same. Say that s; and s; are two 
such identical states, with 0 < i < j < N. This means that f(s;, 0°) = sj, where t = j —i. It 
follows that there is a loop leading from s; back to itself, obtained using the input 0 a total of ¢ 
times, in the state diagram shown in Figure 6. 

Now consider the input string 0“ 0'1" = 0+'1. There are ¢ more consecutive Os at the 
start of this block than there are consecutive Is that follow it. Because this string is not of the 
form 0”1” (because it has more Os than Is), it is not recognized by M. Consequently, 
f(so,0%*‘1%) cannot be a final state. However, when we use the string 0ON+'1% as input, 
we end up in the same state as before, namely, s2,. The reason for this is that the extra ft Os in 
this string take us around the loop from s; back to itself an extra time, as shown in Figure 6. 
Then the rest of the string leads us to exactly the same state as before. This contradiction shows 
that {0"1”" |n = 0, 1, 2,...} is not regular. < 


More Powerful Types of Machines 


Finite-state automata are unable to carry out many computations. The main limitation of these 
machines is their finite amount of memory. This prevents them from recognizing languages that 
are not regular, such as {0"1” | n = 0, 1, 2,...}. Because a set is regular if and only if it is the 
language generated by a regular grammar, Example 6 shows that there is no regular grammar 
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that generates the set {0"1” | n = 0, 1,2, ...}. However, there is a context-free grammar that 
generates this set. Such a grammar was given in Example 5 in Section 13.1. 

Because of the limitations of finite-state machines, it is necessary to use other, more pow- 
erful, models of computation. One such model is the pushdown automaton. A pushdown 
automaton includes everything in a finite-state automaton, as well as a stack, which provides 
unlimited memory. Symbols can be placed on the top or taken off the top of the stack. A set is 
recognized in one of two ways by a pushdown automaton. First, a set is recognized if the set 
consists of all the strings that produce an empty stack when they are used as input. Second, a 
set is recognized if it consists of all the strings that lead to a final state when used as input. It 
can be shown that a set is recognized by a pushdown automaton if and only if it is the language 
generated by a context-free grammar. 

However, there are sets that cannot be expressed as the language generated by a context- 
free grammar. One such set is {0”1"2” | n = 0, 1, 2, ...}. We will indicate why this set cannot 
be recognized by a pushdown automaton, but we will not give a proof, because we have not 
developed the machinery needed. (However, one method of proof is given in Exercise 28 of the 
supplementary exercises at the end of this chapter.) The stack can be used to show that a string 
begins with a sequence of Os followed by an equal number of Is by placing a symbol on the 
stack for each 0 (as long as only Os are read), and removing one of these symbols for each 1 (as 
long as only Is following the Os are read). But once this is done, the stack is empty, and there 
is no way to determine that there are the same number of 2s in the string as Os. 

There are other machines called linear bounded automata, more powerful than push- 
down automata, that can recognize sets such as {0"1"2” |n = 0,1, 2, ...}. In particular, linear 
bounded automata can recognize context-sensitive languages. However, these machines cannot 
recognize all the languages generated by phrase-structure grammars. To avoid the limitations 
of special types of machines, the model known as a Turing machine, named after the British 
mathematician Alan Turing, is used. A Turing machine is made up of everything included in a 
computers finite-state machine together with a tape, which is infinite in both directions. A Turing machine 

has read and write capabilities on the tape, and it can move back and forth along this tape. Tur- 
ing machines can recognize all languages generated by phrase-structure grammars. In addition, 
Turing machines can model all the computations that can be performed on a computing ma- 
chine. Because of their power, Turing machines are extensively studied in theoretical computer 
science. We will briefly study them in Section 13.5. 


ALAN MATHISON TURING (1912-1954) Alan Turing was born in London, although he was conceived in 
India, where his father was employed in the Indian Civil Service. As a boy, he was fascinated by chemistry, 
performing a wide variety of experiments, and by machinery. Turing attended Sherborne, an English boarding 
school. In 1931 he won a scholarship to King’s College, Cambridge. After completing his dissertation, which 
included a rediscovery of the central limit theorem, a famous theorem in statistics, he was elected a fellow of 
his college. In 1935 Turing became fascinated with the decision problem, a problem posed by the great German 
mathematician Hilbert, which asked whether there is a general method that can be applied to any assertion to 
1 determine whether the assertion is true. Turing enjoyed running (later in life running as a serious amateur in 
+ competitions), and one day, while resting after a run, he discovered the key ideas needed to solve the decision 
problem. In his solution, he invented what is now called a Turing machine as the most general model of a computing machine. 
Using these machines, he found a problem, involving what he called computable numbers, that could not be decided using a general 
method. 

From 1936 to 1938 Turing visited Princeton University to work with Alonzo Church, who had also solved Hilbert’s decision 
problem. In 1939 Turing returned to King’s College. However, at the outbreak of World War II, he joined the Foreign Office, 
performing cryptanalysis of German ciphers. His contribution to the breaking of the code of the Enigma, a mechanical German 
cipher machine, played an important role in winning the war. 

After the war, Turing worked on the development of early computers. He was interested in the ability of machines to think, 
proposing that if a computer could not be distinguished from a person based on written replies to questions, it should be considered 
to be “thinking.” He was also interested in biology, having written on morphogenesis, the development of form in organisms. In 1954 
Turing committed suicide by taking cyanide, without leaving a clear explanation. Legal troubles related to a homosexual relationship 
and hormonal treatments mandated by the court to lessen his sex drive may have been factors in his decision to end his life. 


Exercises 


10. 


. Describe in words the strings in each of these regular sets. 


a) 1*0 b) 1*00* 
c) 111U 001 d) (1U00)* 
e) (00*1)* f) (0U1)0U1)*00 
. Describe in words the strings in each of these regular sets. 
a) 001* b) (01)* 
c) 01U 001* d) 0(11U0)* 
e) (101*)* f) (O*U1)11 
. Determine whether 0101 belongs to each of these regular 
sets. 
a) 01*0* b) 0(11)*(01)* 


c) 0(10)*1* 
e) (01)*(1)* 
g) 0*(10)*11 


d) 0*10(0U 1) 
f) 0*(10U11)* 
h) 01(01 U0)1* 


. Determine whether 1011 belongs to each of these regular 


sets. 

a) 10*1* 

c) 1(01)*1* 
e) (10)*(11)* 
g) (10)*1011 


b) 0*(10 U 11)* 

d) 1*010U1) 

f) 1(00)*(11)* 

h) (1U00)(01 U 0)1* 


. Express each of these sets using a regular expression. 


a) the set consisting of the strings 0, 11, and 010 

b) the set of strings of three Os followed by two or 
more Os 

c) the set of strings of odd length 

d) the set of strings that contain exactly one 1 

e) the set of strings ending in | and not containing 000 


. Express each of these sets using a regular expression. 


a) the set containing all strings with zero, one, or two 
bits 

b) the set of strings of two Os, followed by zero or 
more Is, and ending with a 0 

c) the set of strings with every | followed by two Os 

d) the set of strings ending in 00 and not containing 11 

e) the set of strings containing an even number of Is 


. Express each of these sets using a regular expression. 


a) the set of strings of one or more Os followed by a 1 

b) the set of strings of two or more symbols followed by 
three or more Os 

c) the set of strings with either no 1 preceding a 0 or 
no 0 preceding a | 

d) the set of strings containing a string of 1s such that 
the number of Is equals 2 modulo 3, followed by an 
even number of Os 


. Construct deterministic finite-state automata that recog- 


nize each of these sets from /*, where J is an alphabet. 


a) 0 b) {A} c) {a}, where a € I 


. Construct nondeterministic finite-state automata that rec- 


ognize each of the sets in Exercise 8. 


Construct nondeterministic finite-state automata that rec- 
ognize each of these sets. 


a) {A, 0} b) {0, 11} c) {0, 11, 000} 


*11. 


12. 


13. 


14. 


13.4 Language Recognition 887 


Show that if A is a regular set, then A®, the set of all 

reversals of strings in A, is also regular. 

Using the constructions described in the proof of Kleene’s 

theorem, find nondeterministic finite-state automata that 

recognize each of these sets. 

a) 01* b) OU1)1* c) 00(1* U 10) 

Using the constructions described in the proof of Kleene’s 

theorem, find nondeterministic finite-state automata that 

recognize each of these sets. 

a) 0*1* b) (OU 11)* c) 01* U 00*1 

Construct a nondeterministic finite-state automaton that 

recognizes the language generated by the regular gram- 

mar G = (V,T,S, P), where V = {0,1, S$, A, B}, T= 

{0, 1}, S is the start symbol, and the set of productions is 

a) S>0A,S>1B,A>0,B > 0. 

b) S> 1A, S>0, S>i, A>OB, B> 1B, 
Bo 1. 

ec) S> 1B, S>0, A>1A, A->OB, Al, 
A> 0,B->1. 


In Exercises 15-17 construct a regular grammar G = 
(V,T,S, P) that generates the language recognized by the 
given finite-state machine. 


15. 


16. 


17. 


18. 


19. 


Start 
1 


Start 
Start 1 
“)+-@O: 
Show that the finite-state automaton constructed from a 


regular grammar in the proof of Theorem 2 recognizes 
the set generated by this grammar. 
Show that the regular grammar constructed from a finite- 
state automaton in the proof of Theorem 2 generates the 
set recognized by this automaton. 
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20. 


*21. 


* 22. 


* 23. 


* 24, 


* 25, 


** 26, 


13 / Modeling Computation 


Show that every nondeterministic finite-state automaton 
is equivalent to another such automaton that has the prop- 
erty that its starting state is never revisited. 

Let M = (S,I, f, 59, F) be a deterministic finite-state 
automaton. Show that the language recognized by 
M, L(M), is infinite if and only if there is a word x rec- 
ognized by M with /(x) > |S}. 

One important technique used to prove that certain sets are 
not regular is the pumping lemma. The pumping lemma 
states thatif M = (S, /, f, so, F) isa deterministic finite- 
state automaton and if x is a string in L(M), the language 
recognized by M, with /(x) > |S|, then there are strings 
u, v, and w in J* such that x = uvw, I(uv) < |S| and 
I(v) = 1, and uv'w € L(M) for i =0,1,2,.... Prove 
the pumping lemma. [Hint: Use the same idea as was 
used in Example 5.] 

Show that the set 02" 1” |n =0, 1, 2,...} is not regular 
using the pumping lemma given in Exercise 22. 

Show that the set qe |n =0,1,2,...} is not regular 
using the pumping lemma from Exercise 22. 

Show that the set of palindromes over {0, 1} is not reg- 
ular using the pumping lemma given in Exercise 22. 
[Hint: Consider strings of the form 010%] 

Show that a set recognized by a finite-state automaton is 
regular. (This is the if part of Kleene’s theorem.) 


Suppose that L is a subset of /*, where J is a nonempty set of 
symbols. If x € I*, we let L/x = {z € I* | xz € L}. We say 
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that the strings x € J* and y € /* are distinguishable with 
respect to L if L/x # L/y. A string z for which xz € L but 
yz € L, or xz ¢ L, but yz € Lis said to distinguish x and y 
with respect to L. When L/x = L/y, we say that x and y are 
indistinguishable with respect to L. 


27. 


28. 


* 29, 


* 30. 


* 31. 


Let L be the set of all bit strings that end with 01. Show 
that 11 and 10 are distinguishable with respect to L and 
that the strings 1 and 11 are indistinguishable with respect 
to L. 


Suppose that M = (S,/, f, sq, F) is a deterministic 
finite-state machine. Show that if x and y are two strings 
in /* that are distinguishable with respect to L(M), then 


f (so, x) # f(So, y). 


Suppose that L is a subset of /* and for some positive 
integer n there are n strings in /* such that every two of 
these strings are distinguishable with respect to L. Prove 
that every deterministic finite-state automaton recogniz- 
ing L has at least n states. 


Let L,, be the set of strings with at least n bits in which the 
nth symbol from the end is a 0. Use Exercise 29 to show 
that a deterministic finite-state machine recognizing Ly 
must have at least 2” states. 


Use Exercise 29 to show that the language consisting of 
all bit strings that are palindromes (that is, strings that 
equal their own reversals) is not regular. 


“Machines take me by 
surprise with great 
frequency” — Alan Turing 


Introduction 


Links 


The finite-state automata studied earlier in this chapter cannot be used as general models of 
computation. They are limited in what they can do. For example, finite-state automata are able 


to recognize regular sets, but are not able to recognize many easy-to-describe sets, including 
{0" 1” | n > O}, which computers recognize using memory. We can use finite-state automata to 
compute relatively simple functions such as the sum of two numbers, but we cannot use them to 
compute functions that computers can, such as the product of two numbers. To overcome these 
deficiencies we can use a more powerful type of machine known as a Turing machine, after Alan 
Turing, the famous mathematician and computer scientist who invented them in the 1930s. 
Basically, a Turing machine consists of a control unit, which at any step is in one of finitely 
many different states, together with a tape divided into cells, which is infinite in both directions. 
Turing machines have read and write capabilities on the tape as the control unit moves back and 
forth along this tape, changing states depending on the tape symbol read. Turing machines are 
more powerful than finite-state machines because they include memory capabilities that finite- 
state machines lack. We will show how to use Turing machines to recognize sets, including 


sets that cannot be recognized by finite-state machines. We will also show how to compute 
functions using Turing machines. Turing machines are the most general models of computation; 
essentially, they can do whatever a computer can do. Note that Turing machines are much more 


powerful than real computers, which have finite memory capabilities. 


DEFINITION 1 


EXAMPLE 1 


Extra pe 
Examples 
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Control 
Unit 
~t— Read/Write Head 
B B 1 1 0 1 B 0 1 B B 


Tape is infinite in both directions. 
Only finitely many nonblank cells at any time. 


FIGURE 1 A Representation of a Turing Machine. 


Definition of Turing Machines 


We now give the formal definition of a Turing machine. Afterward we will explain how this 
formal definition can be interpreted in terms of a control head that can read and write symbols 
on a tape and move either right or left along the tape. 


A Turing machine T = (S, I, f, so) consists of a finite set S of states, an alphabet / containing 
the blank symbol B, a partial function f from S x J toS x I x {R, L}, and a starting state sq. 


Recall from Section 2.3 that a partial function is defined only for those elements in its domain 
of definition. This means that for some (state, symbol) pairs the partial function f may be 
undefined, but for a pair for which it is defined, there is a unique (state, symbol, direction) 
triple associated to this pair. We call the five-tuples corresponding to the partial function in the 
definition of a Turing machine the transition rules of the machine. 

To interpret this definition in terms of a machine, consider a control unit and a tape divided 
into cells, infinite in both directions, having only a finite number of nonblank symbols on it at 
any given time, as pictured in Figure |. The action of the Turing machine at each step of its 
operation depends on the value of the partial function f for the current state and tape symbol. 

At each step, the control unit reads the current tape symbol x. If the control unit is in state s 
and if the partial function f is defined for the pair (s, x) with f(s, x) = (s’, x’, d), the control 
unit 


1. enters the state s’, 
2. writes the symbol x’ in the current cell, erasing x, and 
3. moves right one cell if d = R or moves left one cell if d = L. 


We write this step as the five-tuple (s, x, s’, x’, d). If the partial function f is undefined for the 
pair (s, x), then the Turing machine T will halt. 

A common way to define a Turing machine is to specify a set of five-tuples of the form 
(s, x, s’, x’, d). The set of states and input alphabet is implicitly defined when such a definition 
is used. 

At the beginning of its operation a Turing machine is assumed to be in the initial state so 
and to be positioned over the leftmost nonblank symbol on the tape. If the tape is all blank, the 
control head can be positioned over any cell. We will call the positioning of the control head 
over the leftmost nonblank tape symbol the initial position of the machine. 

Example | illustrates how a Turing machine works. 


What is the final tape when the Turing machine JT defined by the seven five- 
tuples (so, 0, 50,0, R), (So, 1,51, 1, R), (so, B, 53, B, R), (91,0, 50,0, R), (81, 1, 52,0, L) 
(s1, B, 53, B, R), and (s2, 1, 53, 0, R) is run on the tape shown in Figure 2(a)? 
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(a) So Initial Position 
Y 
B B 0 1 0 1 1 0 B B 
(b) 5 
i] 
B B 0 1 0 1 1 0 B B 


(c) Sy 


(d) So 


(e) Sy 


(f) > 


(g) 53 


B B 0 1 0 0 0 0 B B 


Machine Halts 


FIGURE 2 The Steps Produced by Running T on the Tape in Figure 1. 


Solution: We start the operation with T in state so and with T positioned over the leftmost 
nonblank symbol on the tape. The first step, using the five-tuple (so, 0, 59, 0, R), reads the 0 in 
the leftmost nonblank cell, stays in state sg, writes a 0 in this cell, and moves one cell right. 
The second step, using the five-tuple (so, 1, 51, 1, R), reads the | in the current cell, enters 
state s;, writes a | in this cell, and moves one cell right. The third step, using the five-tuple 
(s,;, 0, 59, 0, R), reads the O in the current cell, enters state sg, writes a 0 in this cell, and moves 
one cell right. The fourth step, using the five-tuple (so, 1, 51, 1, R), reads the | in the current 
cell, enters state s;, writes a | in this cell, and moves right one cell. The fifth step, using the 
five-tuple (s;, 1, s2, 0, L), reads the | in the current cell, enters state 52, writes a 0 in this cell, 
and moves left one cell. The sixth step, using the five-tuple (s2, 1, 53,0, R), reads the | in the 
current cell, enters the state 53, writes a 0 in this cell, and moves right one cell. Finally, in the 
seventh step, the machine halts because there is no five-tuple beginning with the pair (s3, 0) in 
the description of the machine. The steps are shown in Figure 2. 

Note that T changes the first pair of consecutive 1s on the tape to Os and then halts. < 


DEFINITION 2 


EXAMPLE 2 


EXAMPLE 3 
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Using Turing Machines to Recognize Sets 


Turing machines can be used to recognize sets. To do so requires that we define the concept of 
a final state as follows. A final state of a Turing machine T is a state that is not the first state in 
any five-tuple in the description of T using five-tuples (for example, state s3 in Example 1). 

We can now define what it means for a Turing machine to recognize a string. Given a string, 
we write consecutive symbols in this string in consecutive cells. 


Let V be a subset of an alphabet 7. A Turing machine T = (S, J, f, so) recognizes a string x 
in V* if and only if 7, starting in the initial position when x is written on the tape, halts in 
a final state. T is said to recognize a subset A of V* if x is recognized by T if and only if x 
belongs to A. 


Note that to recognize a subset A of V* we can use symbols not in V. This means that the input 
alphabet J may include symbols not in V. These extra symbols are often used as markers (see 
Example 3). 

When does a Turing machine T not recognize a string x in V*? The answer is that x is 
not recognized if T does not halt or halts in a state that is not final when it operates on a tape 
containing the symbols of x in consecutive cells, starting in the initial position. (The reader 
should understand that this is one of many possible ways to define how to recognize sets using 
Turing machines.) 

We illustrate this concept with Example 2. 


Find a Turing machine that recognizes the set of bit strings that have a | as their second bit, that 
is, the regular set (QU D10U1)*. 


Solution: We want a Turing machine that, starting at the leftmost nonblank tape cell, moves 
right, and determines whether the second symbol is a 1. If the second symbol is 1, the machine 
should move into a final state. If the second symbol is not a 1, the machine should not halt or it 
should halt in a nonfinal state. 

To construct such a machine, we include the five-tuples (so, 0, 51, 0, R) and (so, 1, 51, 1, R) 
to read in the first symbol and put the Turing machine in state s;. Next, we include the five-tuples 
(s1, 0, 82,0, R) and (s;, 1, 53, 1, R) to read in the second symbol and either move to state s2 if 
this symbol is a 0, or to state 53 if this symbol is a 1. We do not want to recognize strings that 
have a 0 as their second bit, so s2 should not be a final state. We want s3 to be a final state. So, we 
can include the five-tuple (s2, 0, 52, 0, R). Because we do not want to recognize the empty string 
or a string with one bit, we also include the five-tuples (so, B, s2, 0, R) and (s1, B, s2, 0, R). 

The Turing machine 7 consisting of the seven five-tuples listed here will terminate in the 
final state s3 if and only if the bit string has at least two bits and the second bit of the input string 
is a 1. If the bit string contains fewer than two bits or if the second bit is not a 1, the machine 
will terminate in the nonfinal state s2. < 


Given a regular set, a Turing machine that always moves to the right can be built to recognize 
this set (as in Example 2). To build the Turing machine, first find a finite-state automaton that 
recognizes the set and then construct a Turing machine using the transition function of the 
finite-state machine, always moving to the right. 

We will now show how to build a Turing machine that recognizes a nonregular set. 


Find a Turing machine that recognizes the set {0”1”" | n > 1}. 
Solution: To build such a machine, we will use an auxiliary tape symbol M as a marker. We have 


V = {0, 1} and J = {0, 1, M}. We wish to recognize only a subset of strings in V*. We will 
have one final state, s6. The Turing machine successively replaces a 0 at the leftmost position of 


892 


13 / Modeling Computation 


EXAMPLE 4 


the string with an M and a | at the rightmost position of the string with an M, sweeping back 
and forth, terminating in a final state if and only if the string consists of a block of Os followed 
by a block of the same number of Is. 

Although this is easy to describe and is easily carried out by a Turing machine, the machine 
we need to use is somewhat complicated. We use the marker M to keep track of the leftmost 
and rightmost symbols we have already examined. The five-tuples we use are (so, 0, 51, M, R), 
(s},0,513,0,R), (51,1,51,1, R), (81,M,50,M,L), (51, B,52,B,L), (52,1,53,M,L), 
(s3, 1, 53,1, L), (83,0, 54,0, L), (s3,M,55,M,R), (54,0, 54,0, L), (84, M,59,M,R), and 
(s5, M,s6,M,R). For example, the string 000111 would successively become MO00111, 
MO011M, MMO11M, MM0IMM, MMM1MM, MMMMM\M as the machine operates 
until it halts. Only the changes are shown, as most steps leave the string unaltered. 

We leave it to the reader (Exercise 13) to explain the actions of this Turing machine and to 
explain why it recognizes the set {0"1” | n > 1}. < 


It can be shown that a set can be recognized by a Turing machine if and only if it can be 
generated by a type 0 grammar, or in other words, if the set is generated by a phrase-structure 
grammar. The proof will not be presented here. 


Computing Functions with Turing Machines 


A Turing machine can be thought of as a computer that finds the values of a partial function. 
To see this, suppose that the Turing machine 7, when given the string x as input, halts with 
the string y on its tape. We can then define T(x) = y. The domain of T is the set of strings 
for which T halts; T(x) is undefined if T does not halt when given x as input. Thinking of a 
Turing machine as a machine that computes the values of a function on strings is useful, but 
how can we use Turing machines to compute functions defined on integers, on pairs of integers, 
on triples of integers, and so on? 

To consider a Turing machine as a computer of functions from the set of k-tuples of non- 
negative integers to the set of nonnegative integers (such functions are called number-theoretic 
functions), we need a way to represent k-tuples of integers on a tape. To do so, we use unary rep- 
resentations of integers. We represent the nonnegative integer n by a string of n + 1 Is so that, 
for instance, 0 is represented by the string | and 5 is represented by the string 111111. To repre- 
sent the k-tuple (nj, n2,...,x), we use a string of n; + 1 1s, followed by an asterisk, followed 
by a string of nz + | 1s, followed by an asterisk, and so on, ending with a string of nz + 1 Is. 
For example, to represent the four-tuple (2, 0, 1, 3) we use the string] 11 * 1 * 11 * 1111. 

We can now consider a Turing machine T as computing a sequence of number-theoretic 
functions T, T?,..., T7*,.... The function T* is defined by the action of T on k-tuples of 
integers represented by unary representations of integers separated by asterisks. 


Construct a Turing machine for adding two nonnegative integers. 


Solution: We need to build a Turing machine T that computes the function f(n1,n2) =n, +n2. 
The pair (n;,2) is represented by a string of n; +1 Is followed by an asterisk followed 
by n2 +1 Is. The machine T should take this as input and produce as output a tape with 
nj +n2+1 Is. One way to do this is as follows. The machine starts at the leftmost 1 of the 
input string, and carries out steps to erase this |, halting if 2, = O so that there are no more Is 
before the asterisk, replaces the asterisk with the leftmost remaining 1, and then halts. We can use 
these five-tuples to do this: (so, 1, 51, B, R), (81, *, 83, B, R), (81, 1, 52, B, R), (s2, 1, 52, 1, 
and (s2, *, 53, 1, R). 


Unfortunately, constructing Turing machines to compute relatively simple functions can 
be extremely demanding. For example, one Turing machine for multiplying two nonnegative 
integers found in many books has 31 five-tuples and 11 states. If it is challenging to construct 
Turing machines to compute even relatively simple functions, what hope do we have of building 
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Turing machines for more complicated functions? One way to simplify this problem is to use a 
multitape Turing machine that uses more than one tape simultaneously and to build up multitape 
Turing machines for the composition of functions. It can be shown that for any multitape Turing 
machine there is a one-tape Turing machine that can do the same thing. 


Different Types of Turing Machines 


There are many variations on the definition of a Turing machine. We can expand the capabilities 
of a Turing machine in a wide variety of ways. For example, we can allow a Turing machine to 
move right, left, or not at all at each step. We can allow a Turing machine to operate on multiple 
tapes, using (2 + 37)-tuples to describe the Turing machine when n tapes are used. We can allow 
the tape to be two-dimensional, where at each step we move up, down, right, or left, not just right 
or left as we do on a one-dimensional tape. We can allow multiple tape heads that read different 
cells simultaneously. Furthermore, we can allow a Turing machine to be nondeterministic, by 
allowing a (state, tape symbol) pair to possibly appear as the first elements in more than one 
five-tuple of the Turing machine. We can also reduce the capabilities of a Turing machine in 
different ways. For example, we can restrict the tape to be infinite in only one dimension or we 
can restrict the tape alphabet to have only two symbols. All these variations of Turing machines 
have been studied in detail. 

The crucial point is that no matter which of these variations we use, or even which combi- 
nation of variations we use, we never increase or decrease the power of the machine. Anything 
that one of these variations can do can be done by the Turing machine defined in this section, 
and vice versa. The reason that these variations are useful is that sometimes they make doing 
some particular job much easier than if the Turing machine defined in Definition | were used. 
They never extend the capability of the machine. Sometimes it is useful to have a wide variety 
of Turing machines with which to work. For example, one way to show that for every nonde- 
terministic Turing machine, there is a deterministic Turing machine that recognizes the same 
language is to use a deterministic Turing machine with three taps. (For details on variations of 
Turing machines and demonstrations of their equivalence, see [HoMoUIO1].) 

Besides introducing the notion of a Turing machine, Turing also showed that it is possible to 
construct a single Turing machine that can simulate the computations of every Turing machine 
when given an encoding of this target Turing machine and its input. Such a machine is called a 
universal Turing machine. (See a book on the theory of computation, such as [S106], for more 
about universal Turing machines.) 


The Church-Turing Thesis 


Turing machines are relatively simple. They can have only finitely many states and they can 
read and write only one symbol at a time on a one-dimensional tape. But it turns out that Turing 
machines are extremely powerful. We have seen that Turing machines can be built to add numbers 
and to multiply numbers. Although it may be difficult to actually construct a Turing machine to 
compute a particular function that can be computed with an algorithm, such a Turing machine 
can always be found. This was the original goal of Turing when he invented his machines. 
Furthermore, there is a tremendous amount of evidence for the Church-Turing thesis, which 
states that given any problem that can be solved with an effective algorithm, there is a Turing 
machine that can solve this problem. The reason this is called a thesis rather than a theorem is that 
the concept of solvability by an effective algorithm is informal and imprecise, as opposed to the 
notion of solvability by a Turing machine, which is formal and precise. Certainly, though, any 
problem that can be solved using a computer with a program written in any language, perhaps 
using an unlimited amount of memory, should be considered effectively solvable. (Note that 
Turing machines have unlimited memory, unlike computers in the real world, which have only 
a finite amount of memory.) 
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Many different formal theories have been developed to capture the notion of effective 
computability. These include Turing’s theory and Church’s lambda-calculus, as well as theories 
proposed by Stephen Kleene and by E. L. Post. These theories seem quite different on the surface. 
The surprising thing is that they can be shown to be equivalent by demonstrating that they define 
exactly the same class of functions. With this evidence, it seems that Turing’s original ideas, 
formulated before the invention of modern computers, describe the ultimate capabilities of these 
machines. The interested reader should consult books on the theory of computation, such as 
[HoMoUI101] and [S196], for a discussion of these different theories and their equivalence. 

For the remainder of this section we will briefly explore some of the consequences of the 
Church—Turing thesis and we will describe the importance of Turing machines in the study of the 
complexity of algorithms. Our goal will be to introduce some important ideas from theoretical 
computer science to entice the interested student to further study. We will cover a lot of ground 
quickly without providing explicit details. Our discussion will also tie together some of the 
concepts discussed in previous parts of the book with the theory of computation. 


Computational Complexity, Computability, and Decidability 


Throughout this book we have discussed the computational complexity of a wide variety of 
problems. We described the complexity of these problems in terms of the number of operations 
used by the most efficient algorithms that solve them. The basic operations used by algorithms 
differ considerably; we have measured the complexity of different algorithms in terms of bit 
operations, comparisons of integers, arithmetic operations, and so on. In Section 3.3, we defined 
various classes of problems in terms of their computational complexity. However, these defini- 
tions were not precise, because the types of operations used to measure their complexity vary so 
drastically. Turing machines provide a way to make the concept of computational complexity 
precise. If the Church—Turing thesis is true, it would then follow that if a problem can be solved 
using an effective algorithm, then there is a Turing machine that can solve this problem. When 
a Turing machine is used to solve a problem, the input to the problem is encoded as a string 
of symbols that is written on the tape of this Turing machine. How we encode input depends 
on the domain of this input. For example, as we have seen, we can encode a positive integer 
using a string of 1s. We can also devise ways to express pairs of integers, negative integers, and 
so on. Similarly, for graph algorithms, we need a way to encode graphs as strings of symbols. 
This can be done in many ways and can be based on adjacency lists or adjacency matrices. 
(We omit the details of how this is done.) However, the way input is encoded does not matter 
as long as it is relatively efficient, as a Turing machine can always change one encoding into 
another encoding. We will now use this model to make precise some of the notions concerning 
computational complexity that were informally introduced in Section 3.3. 

The kind of problems that are most easily studied by using Turing machines are those 
problems that can be answered either by a “yes” or by a “no.” 


A decision problem asks whether statements from a particular class of statements are true. 
Decision problems are also known as yes-or-no problems. 


Given a decision problem, we would like to know whether there is an algorithm that can 
determine whether statements from the class of statements it addresses are true. For example, 
consider the class of statements each of which asks whether a particular integer n is prime. 
This is a decision problem because the answer to the question “Is prime?” is either yes or no. 
Given this decision problem, we can ask whether there is an algorithm that can decide whether 
each of the statements in the decision problem is true, that is, given an integer n, deciding 
whether n is prime. The answer is that there is such an algorithm. In particular, in Section 3.5 
we discussed the algorithm that determines whether a positive integer n is prime by checking 
whether it is divisible by primes not exceeding its square root. (There are many other algorithms 
for determining whether a positive integer is prime.) The set of inputs for which the answer to 
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the yes—no problem is “yes” is a subset of the set of possible inputs, that is, it is a subset of the 
set of strings of the input alphabet. In other words, solving a yes—no problem is the same as 
recognizing the language consisting of all bit strings that represent input values to the problem 
leading to the answer “yes.” Consequently, solving a yes—no problem is the same as recognizing 
the language corresponding to the input values for which the answer to the problem is “yes.” 


DECIDABILITY When there is an effective algorithm that decides whether instances of a 
decision problem are true, we say that this problem is solvable or decidable. For instance, the 
problem of determining whether a positive integer is prime is a solvable problem. However, if no 
effective algorithm exists for solving a problem, then we say the problem is unsolvable or un- 
decidable. To show that a decision problem is solvable we need only construct an algorithm that 
can determine whether statements of the particular class are true. On the other hand, to show 
that a decision problem is unsolvable we need to prove that no such algorithm exists. (The fact 
that we tried to find such an algorithm but failed, does not prove the problem is unsolvable.) 

By studying only decision problems, it may seem that we are studying only a small set of 
problems of interest. However, most problems can be recast as decision problems. Recasting the 
types of problems we have studied in this book as decision problems can be quite complicated, 
so we will not go into the details of this process here. The interested reader can consult references 
on the theory of computation, such as [Wo87], which, for example, explains how to recast the 
traveling salesperson problem (described in Section 9.6) as a decision problem. (To recast the 
traveling salesman problem as a decision problem, we first consider the decision problem that 
asks whether there is a Hamilton circuit of weight not exceeding k, where k is a positive integer. 
With some additional effort it is possible to use answers to this question for different values 
of k to find the smallest possible weight of a Hamilton circuit.) 

In Section 3.1 we introduced the halting problem and proved that it is an unsolvable problem. 
That discussion was somewhat informal because the notion of a procedure was not precisely 
defined. A precise definition of the halting problem can be made in terms of Turing machines. 


The halting problem is the decision problem that asks whether a Turing machine T eventually 
halts when given an input string x. 


With this definition of the halting problem, we have Theorem 1. 


The halting problem is an unsolvable decision problem. That is, no Turing machine exists 
that, when given an encoding of a Turing machine 7 and its input string x as input, can 
determine whether T eventually halts when started with x written on its tape. 


The proof of Theorem | given in Section 3.1 for the informal definition of the halting problem 
still applies here. 
Other examples of unsolvable problems include: 


(i) the problem of determining whether two context-free grammars generate the same set 

of strings; 

(ii) the problem of determining whether a given set of tiles can be used with repetition 
allowed to cover the entire plane without overlap; and 

(iii) Hilbert’s Tenth Problem, which asks whether there are integer solutions to a given 
polynomial equation with integer coefficients. (This question occurs tenth on the famous 
list of 23 problems Hilbert posed in 1900. Hilbert envisioned that the work done to 
solve these problems would help further the progress of mathematics in the twentieth 
century. The unsolvability of Hilbert’s Tenth Problem was established in 1970 by Yuri 
Matiyasevich.) 
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Links 


DEFINITION 5 


COMPUTABILITY A function that can be computed by a Turing machine is called com- 
putable and a function that cannot be computed by a Turing machine is called uncomputable. 
Itis fairly straightforward, using a countability argument, to show that there are number-theoretic 
functions that are not computable (see Exercise 39 in Section 2.5). However, it is not so easy 
to actually produce such a function. The busy beaver function defined in the preamble to Ex- 
ercise 31 is an example of an uncomputable function. One way to show that the busy beaver 
function is not computable is to show that it grows faster than any computable function. (See 
Exercise 32.) 

Note that every decision problem can be reformulated as the problem of computing a 
function, namely, the function that has the value 1 when the answer to the problem is “yes” and 
that has the value 0 when the answer to the problem is “no.” A decision problem is solvable if 
and only if the corresponding function constructed in this way is computable. 


THE CLASSESPAND NP _ InSection 3.3 we informally defined the classes of problems called 
P and NP. We are now able to define these concepts precisely using the notions of deterministic 
and nondeterministic Turing machines. 

We first elaborate on the difference between a deterministic Turing machine and a nonde- 
terministic Turing machine. The Turing machines we have studied in this section have all been 
deterministic. In a deterministic Turing machine T = (S, /, f, so), transition rules are defined 
by the partial function f from S x J to S x I x {R, L}. Consequently, when transition rules 
of the machine are represented as five-tuples of the form (s, x, s’, x’, d), where s is the current 
state, x is the current tape symbol, s’ is the next state, x’ is the symbol that replaces x on the 
tape, and d is the direction the machine moves on the tape, no two transition rules begin with 
the same pair (s, x). 

In a nondeterministic Turing machine, allowed steps are defined using a relation consisting 
of five-tuples rather than using a partial function. The restriction that no two transition rules 
begin with the same pair (s, x) is eliminated; that is, there may be more than one transition 
rule beginning with each (state, tape symbol) pair. Consequently, in a nondeterministic Turing 
machine, there is a choice of transitions for some pairs of the current state and the tape symbol 
being read. At each step of the operation of a nondeterministic Turing machine, the machine 
picks one of the different choices of the transition rules that begin with the current state and 
tape symbol pair. This choice can be considered to be a “guess” of which step to use. Just 
as for deterministic Turing machines, a nondeterministic Turing machine halts when there is 
no transition rule in its definition that begins with the current state and tape symbol. Given 
a nondeterministic Turing machine 7, we say that a string x is recognized by T if and only 
if there exists some sequence of transitions of T that ends in a final state when the machine 
starts in the initial position with x written on the tape. The nondeterministic Turing machine T 
recognizes the set A if x is recognized by T if and only if x € A. The nondeterministic Turing 
machine T is said to solve a decision problem if it recognizes the set consisting of all input 
values for which the answer to the decision problem is yes. 


A decision problem is in P, the class of polynomial-time problems, if it can be solved by a 
deterministic Turing machine in polynomial time in terms of the size of its input. That is, a 
decision problem is in P if there is a deterministic Turing machine T that solves the decision 
problem and a polynomial p(n) such that for all integers n, T halts in a final state after no 
more than p(7) transitions whenever the input to 7 is a string of length n. A decision problem 
is in NP, the class of nondeterministic polynomial-time problems, if it can be solved by a 
nondeterministic Turing machine in polynomial time in terms of the size of its input. That 
is, a decision problem is in NP if there is a nondeterministic Turing machine T that solves 
the problem and a polynomial p(n) such that for all integers n, T halts for every choice of 
transitions after no more than p() transitions whenever the input to 7 is a string of length n. 
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Problems in P are called tractable, whereas problems not in P are called intractable. For 
a problem to be in P, a deterministic Turing machine must exist that can decide in polynomial 
time whether a particular statement of the class addressed by the decision problem is true. For 
example, determining whether an item is in a list of n elements is a tractable problem. (We 
will not provide details on how this fact can be shown; the basic ideas used in the analyses 
of algorithms earlier in the text can be adapted when Turing machines are employed.) For a 
problem to be in NP, it is necessary only that there be a nondeterministic Turing machine that, 
when given a true statement from the set of statements addressed by the problem, can verify its 
truth in polynomial time by making the correct guess at each step from the set of allowable steps 
corresponding to the current state and tape symbol. The problem of determining whether a given 
graph has a Hamilton circuit is an NP problem, because a nondeterministic Turing machine can 
easily verify that a simple circuit in a graph passes through each vertex exactly once. It can 
do this by making a series of correct guesses corresponding to successively adding edges to 
form the circuit. Because every deterministic Turing machine can also be considered to be a 
nondeterministic Turing machine where each (state, tape symbol) pair occurs in exactly one 
transition rule defining the machine, every problem in P is also in NP. In symbols, P € NP. 

One of the most perplexing open questions in theoretical computer science is whether every 
problem in NP is also in P, that is, whether P = NP. As mentioned in Section 3.3, there is an 
important class of problems, the class of NP-complete problems, such that a problem is in this 
class if itis in the class NP and if it can be shown that if it is also in the class P, then every problem 
in the class NP must also be in the class P. That is, a problem is NP-complete if the existence of 
a polynomial-time algorithm for solving it implies the existence of a polynomial-time algorithm 
for every problem in NP. In this book we have discussed several different NP-complete problems, 
such as determining whether a simple graph has a Hamilton circuit and determining whether a 
proposition in n-variables is a tautology. 


Exercises 
1. Let T be the Turing machine defined by the five- 2. Let T be the Turing machine defined by the five- 
tuples: (so, 0, 51, 1, R), (so, 1, 51, 0, R), (so, B, 51, 0, R), tuples: (so, 0, 51, 0, R), (so, 1, 51, 0, L), (so, B, 51, 1, R), 
(51,0, 52,1, 2), (s1,1,51,0, R), and (51, B, 52,0, L). (51,0, 82,1, R), (51, 1,51,1, R), (61, B, 52,0, R), and 
For each of these initial tapes, determine the final (s2, B, 53,0, R). For each of these initial tapes, deter- 
tape when T halts, assuming that T begins in initial mine the final tape when T halts, assuming that T begins 
position. in initial position. 


a) [ea] elojolitijalal.. a) [eB] Blolifolijalal.. 
b- | al altijoli{alalal-. bd [al eltijifitslalel.. 
o--[a}alililalojilal.. o- [a] slololsiojolsal.. 
d-- |e] a]aislisisl{slsl.. a. |aialaialsaislalal 


Links 


ALONZO CHURCH (1903-1995) Alonzo Church was born in Washington, D.C. He studied at Gottingen 
under Hilbert and later in Amsterdam. He was a member of the faculty at Princeton University from 1927 until 
1967 when he moved to UCLA. Church was one of the founding members of the Association for Symbolic 
Logic. He made many substantial contributions to the theory of computability, including his solution to the 
decision problem, his invention of the lambda-calculus, and, of course, his statement of what is now known 
as the Church-Turing thesis. Among Church’s students were Stephen Kleene and Alan Turing. He published 
articles past his 90th birthday. 
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. What does the Turing machine described by the five- 


tuples (so, 0, 50, 0, R), (so, 1, 51,0, R), (So, B, 52, B, R), 
(51,0, 51,0, R), (s1, 1, 50, 1, R), and (51, B, 52, B, R) do 
when given 

a) 11 as input? 

b) an arbitrary bit string as input? 


. What does the Turing machine described by the five- 


tuples (so, 0, so, 1, R), (So, 1, 50, 1, R), (so, B, 51, B, L), 
(s1, 1, 52, 1, R), do when given 

a) 101 as input? 

b) an arbitrary bit string as input? 


. What does the Turing machine described by the five- 


tuples (sg, 1, 51,0, R), (1, 1, 51, 1, R), (51, 0, 52, 0, R), 
(s2, 0, 53, 1, L), (s2, 1, s2, 1, R), (s3, 1, 53, 1, L), 
(s3, 0, 54,0, L), (84, 1, s4, 1, L), and (s4, 0, s9, 1, R) do 
when given 

a) 11 as input? 

b) a bit string consisting entirely of 1s as input? 


. Construct a Turing machine with tape symbols 0, 1, 


and B that, when given a bit string as input, adds a 1 
to the end of the bit string and does not change any of the 
other symbols on the tape. 


. Construct a Turing machine with tape symbols 0, 1, 


and B that, when given a bit string as input, replaces 
the first 0 with a 1 and does not change any of the other 
symbols on the tape. 


. Construct a Turing machine with tape symbols 0, 1, 


and B that, given a bit string as input, replaces all Os 
on the tape with 1s and does not change any of the Is on 
the tape. 


. Construct a Turing machine with tape symbols 0, 1, 


and B that, given a bit string as input, replaces all but 
the leftmost 1 on the tape with Os and does not change 
any of the other symbols on the tape. 


Construct a Turing machine with tape symbols 0, 1, 
and B that, given a bit string as input, replaces the first two 
consecutive 1s on the tape with Os and does not change 
any of the other symbols on the tape. 


Construct a Turing machine that recognizes the set of all 
bit strings that end with a 0. 

Construct a Turing machine that recognizes the set of all 
bit strings that contain at least two 1s. 

Construct a Turing machine that recognizes the set of all 
bit strings that contain an even number of 1s. 

Show at each step the contents of the tape of the Turing 
machine in Example 3 starting with each of these strings. 
a) 0011 b) 00011 c) 101100 d) 000111 
Explain why the Turing machine in Example 3 recognizes 
a bit string if and only if this string is of the form 0” 1” 
for some positive integer n. 

Construct a Turing machine that recognizes the set 
{02"1" | n > O}. 


Construct a Turing machine that recognizes the set 
{071"2" | n > O}. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


* 25, 


26. 


Construct a Turing machine that computes the function 
f(a) =n +2 for all nonnegative integers n. 


Construct a Turing machine that computes the function 
f(a) =n—3ifn> 3 and f(n) = 0 forn = 0, 1, 2 for 
all nonnegative integers n. 


Construct a Turing machine that computes the function 
f(n) =n mod 3 for every nonnegative integer n. 


Construct a Turing machine that computes the function 
f(a) =3 ifn > Sand f(n) = Oifn = 0, 1, 2, 3, or 4. 


Construct a Turing machine that computes the function 
f() = 2n for all nonnegative integers n. 


Construct a Turing machine that computes the function 
f (n) = 3n for all nonnegative integers n. 


Construct a Turing machine that computes the function 
f (1,2) = n2 + 2 for all pairs of nonnegative integers 
n{ and n2. 


Construct a Turing machine that computes the function 
f (1,2) = min(n1, 2) for all nonnegative integers n 
and np. 


Construct a Turing machine that computes the function 
f(m,n2) =n, +n2 + 1 for all nonnegative integers n; 
and np. 


Suppose that 7; and 7> are Turing machines with disjoint sets 
of states S$; and Sz and with transition functions f| and fo, 
respectively. We can define the Turing machine 7) 7>, the com- 
posite of 7; and 7, as follows. The set of states of T, T> is 
S; US). T, Ty begins in the start state of T,. It first executes 
the transitions of 7; using /| up to, but not including, the step 
at which 7; would halt. Then, for all moves for which T; halts, 
it executes the same transitions of T; except that it moves to 
the start state of T>. From this point on, the moves of 7; T> are 
the same as the moves of 7). 


27. 


28. 


29. 


30. 


By finding the composite of the Turing machines you 
constructed in Exercises 18 and 22, construct a Turing 
machine that computes the function f (nm) = 2n + 2. 


By finding the composite of the Turing machines you 
constructed in Exercises 18 and 23, construct a Turing 
machine that computes the function f(n) = 3(n + 2) = 
3n + 6. 


Which of the following problems is a decision problem? 

a) What is the smallest prime greater than n? 

b) Is a graph G bipartite? 

c) Given a set of strings, is there a finite-state automaton 
that recognizes this set of strings? 


d) Given a checkerboard and a particular type of poly- 
omino (see Section 1.8), can this checkerboard be tiled 
using polyominoes of this type? 

Which of the following problems is a decision problem? 

a) Is the sequence aj, a2,.. 
increasing order? 


., Qn Of positive integers in 


b) Can the vertices of a simple graph G be colored using 
three colors so that no two adjacent vertices are the 
same color? 


c) What is the vertex of highest degree in a graph G? 


d) Given two finite-state machines, do these machines 
recognize the same language? 


Let B(n) be the maximum number of 1s that a Turing machine 
with n states with the alphabet {1, B} may print on a tape that 
is initially blank. The problem of determining B(n) for partic- 
ular values of n is known as the busy beaver problem. This 
problem was first studied by Tibor Rado in 1962. Currently it 
is known that B(2) = 4, B(3) = 6, and B(4) = 13, but B(n) 


Key Terms and Results 


TERMS 


alphabet (or vocabulary): a set that contains elements used 
to form strings 

language: a subset of the set of all strings over an alphabet 

phrase-structure grammar (V, 7, S, P): a description of a lan- 
guage containing an alphabet V, a set of terminal symbols 
T, a start symbol S, and a set of productions P 

the production w — wy: w can be replaced by w; whenever 
it occurs in a string in the language 

W1 = W2 (W2 is directly derivable from w 1): w2 can be ob- 
tained from w, using a production to replace a string in w) 
with another string 


W1 => W2 (W2 is derivable from w1): w2 can be obtained from 
w 1 using a sequence of productions to replace strings by 
other strings 

type 0 grammar: any phrase-structure grammar 

type 1 grammar: a phrase-structure grammar in which every 
production is of the form w; — w2, where w; = /Ar and 
w2 = lwr, where A € N,1,7r, w € (N UT) and w 4 A, or 
w, = S and w2 = 2 as long as S is not on the right-hand 
side of another production 

type 2, or context-free, grammar: a phrase-structure gram- 
mar in which every production is of the form A > wy, 
where A is a nonterminal symbol 

type 3, or regular, grammar: a phrase-structure grammar 
where every production is of the form A — aB, A > a, or 
S — 4, where A and B are nonterminal symbols, S is the 
start symbol, and a is a terminal symbol 

derivation (or parse) tree: an ordered rooted tree where the 
root represents the starting symbol of a type 2 grammar, 
internal vertices represent nonterminals, leaves represent 
terminals, and the children of a vertex are the symbols on 
the right side of a production, in order from left to right, 
where the symbol represented by the parent is on the left- 
hand side 

Backus-Naur form: a description of a context-free grammar 
in which all productions having the same nonterminal as 
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is not known for n > 5. B(n) grows rapidly; it is known that 
B(5) > 4098 and B(6) > 3.5 x 1018767, 


* 31. Show that B(2) is at least 4 by finding a Turing machine 
with two states and alphabet {1, B} that halts with four 
consecutive 1s on the tape. 


** 32. Show that the function B(m) cannot be computed by any 
Turing machine. [Hint: Assume that there is a Turing ma- 
chine that computes B(n) in binary. Build a Turing ma- 
chine T that, starting with a blank tape, writes n down 
in binary, computes B(n) in binary, and converts B(n) 
from binary to unary. Show that for sufficiently large n, 
the number of states of T is less than B(n), leading to a 


contradiction. ] 


their left-hand side are combined with the different right- 
hand sides of these productions, each separated by a bar, 
with nonterminal symbols enclosed in angular brackets and 
the symbol — replaced by ::= 

finite-state machine (S, 7, O, f, g, 59) (or a Mealy ma- 
chine): a six-tuple containing a set S of states, an input 
alphabet 7, an output alphabet O, a transition function f 
that assigns a next state to every pair of a state and an input, 
an output function g that assigns an output to every pair of 
a state and an input, and a starting state sq 

AB (concatenation of A and B): the set of all strings formed 
by concatenating a string in A and a string in B in that order 

A* (Kleene closure of A): the set of all strings made up by 
concatenating arbitrarily many strings from A 

deterministic finite-state automaton (S, 7, f, so, F): a five- 
tuple containing a set S of states, an input alphabet J, a 
transition function f that assigns a next state to every pair 
of a state and an input, a starting state so, and a set of final 
states F 

nondeterministic finite-state automaton (S, /, f, 59, F): a 
five-tuple containing a set S of states, an input alphabet J, 
a transition function f that assigns a set of possible next 
states to every pair of a state and an input, a starting state 
So, and a set of final states F 

language recognized by an automaton: the set of input strings 
that take the start state to a final state of the automaton 

regular expression: an expression defined recursively by spec- 
ifying that 4, A, and x, for all x in the input alphabet, are 
regular expressions, and that (AB), (A UB), and A* are 
regular expressions when A and B are regular expressions 

regular set: a set defined by aregular expression (see page 820) 

Turing machine T = (S, J, f, so): afour-tuple consisting of a 
finite set S of states, an alphabet J containing the blank sym- 
bol B, a partial function f from S$ x J to S x I x {R, L}, 
and a starting state so 

nondeterministic Turing machine: a Turing machine that 
may have more than one transition rule corresponding to 
each (state, tape symbol) pair 
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decision problem: a problem that asks whether statements 
from a particular class of statements are true 

solvable problem: a problem with the property that there is 
an effective algorithm that can solve all instances of the 
problem 

unsolvable problem: a problem with the property that no ef- 
fective algorithm exists that can solve all instances of the 
problem 

computable function: a function whose values can be com- 
puted using a Turing machine 

uncomputable function: a function whose values cannot be 
computed using a Turing machine 

P, the class of polynomial-time problems: the class of prob- 
lems that can be solved by a deterministic Turing machine 
in polynomial time in terms of the size of the input 

NP, the class of nondeterministic polynomial-time prob- 
lems: the class of problems that can be solved by a nonde- 


Review Questions 


1. a) Define a phrase-structure grammar. 
b) What does it mean for a string to be derivable from a 
string w by a phrase-structure grammar G? 
2. a) What is the language generated by a phrase-structure 
grammar G? 


b) Whatis the language generated by the grammar G with 
vocabulary {S, 0, 1}, set of terminals T = {0, 1}, start- 
ing symbol S, and productions $ — 000S, S > 1? 


c) Give a phrase-structure grammar that generates the set 
{01” |n =0,1,2,...}. 
3. a) Define a type 1 grammar. 


b) Give an example of a grammar that is not a type 1 
grammar. 


c) Define a type 2 grammar. 


d) Give an example of a grammar that is not a type 2 
grammar but is a type 1 grammar. 


e) Define a type 3 grammar. 


f) Give an example of a grammar that is not a type 3 
grammar but is a type 2 grammar. 


4. a) Define a regular grammar. 
b) Define a regular language. 
c) Show that the set {0 1” | m,n =0,1,2,...}isareg- 
ular language. 
5. a) What is Backus—Naur form? 


b) Give an example of the Backus—Naur form of the 
grammar for a subset of English of your choice. 


6. a) What is a finite-state machine? 


b) Show how a vending machine that accepts only quar- 
ters and dispenses a soft drink after 75 cents has been 
deposited can be modeled using a finite-state machine. 


terministic Turing machine in polynomial time in terms of 
the size of the input 

NP-complete: a subset of the class of NP problems with the 
property that if any one of them is in the class P, then all 
problems in NP are in the class P 


RESULTS 


For every nondeterministic finite-state automaton there is a de- 
terministic finite-state automaton that recognizes the same 
set. 

Kleene’s theorem: A set is regular if and only if there is a 
finite-state automaton that recognizes it. 

A set is regular if and only if it is generated by a regular gram- 
mar. 


The halting problem is unsolvable. 


7. Find the set of strings recognized by the deterministic 
finite-state automaton shown here. 


1 
1 
S 


8. Construct a deterministic finite-state automaton that rec- 
ognizes the set of bit strings that start with | and end 
with 1. 

9. a) What is the Kleene closure of a set of strings? 

b) Find the Kleene closure of the set {11, O}. 
10. a) Define a finite-state automaton. 
b) What does it mean for a string to be recognized by a 
finite-state automaton? 


11. a) Define a nondeterministic finite-state automaton. 

b) Show that given a nondeterministic finite-state au- 
tomaton, there is a deterministic finite-state automaton 
that recognizes the same language. 

12. a) Define the set of regular expressions over a set J. 

b) Explain how regular expressions are used to represent 
regular sets. 

13. State Kleene’s theorem. 

14. Show that a set is generated by a regular grammar if and 
only if it is a regular set. 

15. Give an example of a set not recognized by a finite-state 
automaton. Show that no finite-state automaton recog- 
nizes it. 

16. Define a Turing machine. 

17. Describe how Turing machines are used to recognize sets. 

18. Describe how Turing machines are used to compute 
number-theoretic functions. 

19. What is an unsolvable decision problem? Give an example 
of such a problem. 


Supplementary Exercises 


*1, Find a phrase-structure grammar that generates each of 
these languages. 
a) the set of bit strings of the form 02" 13”, where nis a 
nonnegative integer 
b) the set of bit strings with twice as many Os as Is 
c) the set of bit strings of the form w2, where w is a bit 
string 
*2, Find a phrase-structure grammar that generates the set 
{0?" | n > O}. 
For Exercises 3 and 4, letG = (V, T, S, P) be the context-free 
grammar with V = {(,), $, A, B}, T = {(, )}, starting sym- 
bol S, and productions § ~ A,A — AB,A —> B,B-— (A), 
andB—> (),S >A). 
3. Construct the derivation trees of these strings. 


a)@) b) 0M) ©) (OO) 

*4, Show that L(G) is the set of all balanced strings of paren- 
theses, defined in the preamble to Supplementary Exer- 
cise 55 in Chapter 4. 

A context-free grammar is ambiguous if there is a word in 

L(G) with two derivations that produce different derivation 

trees, considered as ordered, rooted trees. 

5. Show that the grammar G = (V,T,S, P) with V= 
{0,5}, T = {0}, starting state S, and productions § > OS, 
S — SO, and S$ — 0 is ambiguous by constructing two 
different derivation trees for 0°. 

6. Show that the grammar G=(V,T,S, P) with V= 
{0,5}, T = {0}, starting state S, and productions S > 0S 
and S — 0 is unambiguous. 


7. Suppose that A and B are finite subsets of V*, where V 
is an alphabet. Is it necessarily true that |AB| = | BA|? 
8. Prove or disprove each of these statements for subsets A, 
B, and C of V*, where V is an alphabet. 
a) A(BUC) =ABUAC 
b) A(BNC)=ABNAC 
c) (AB)C = A(BC) 
d) (AU B)* = A* U B* 
9. Suppose that A and B are subsets of V*, where V is an 
alphabet. Does it follow that A C B if A* C B*? 
10. What set of strings with symbols in the set {0, 1, 2} is 
represented by the regular expression (2*)(0 U (12*))*? 
The star height (E) of a regular expression over the set J is 
defined recursively by 


h(@O) = 0; 
h(x) = OifxeT/; 
A((E, U Ey)) = h(E) E2)) = max(h(E)), h(E2)) 
if E, and E>» are regular expressions; 
h(E*) = h(E) + 1 if Eis a regular expression. 
11. Find the star height of each of these regular expressions. 
a) 0*1 
b) 0*1* 
c) (0*01)* 
d) ((0*1)*)* 


*12, 


13. 


14. 


15. 


16. 


*17, 


*18. 
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e) (010*)(1*01")* ((01)* (10)*)* 

f) (((((0*)1)*0)*)1)* 

For each of these regular expressions find a regular ex- 
pression that represents the same language with minimum 
star height. 

a) (0* 1*)* 

b) (0(01*0)*)* 

c) (0* U(01)* U1*)* 

Construct a finite-state machine with output that produces 
an output of 1 if the bit string read so far as input contains 
four or more 1s. Then construct a deterministic finite-state 
automaton that recognizes this set. 


Construct a finite-state machine with output that produces 
an output of 1 if the bit string read so far as input contains 
four or more consecutive 1s. Then construct a determin- 
istic finite-state automaton that recognizes this set. 


Construct a finite-state machine with output that produces 
an output of 1 if the bit string read so far as input ends 
with four or more consecutive 1s. Then construct a deter- 
ministic finite-state automaton that recognizes this set. 


A state s’ in a finite-state machine is said to be reach- 
able from state s if there is an input string x such that 
f(s, x) =s'. A state s is called transient if there is no 
nonempty input string x with f(s,x) = s. A state s is 
called a sink if f(s, x) = s for all input strings x. An- 
swer these questions about the finite-state machine with 
the state diagram illustrated here. 


Start 


a) Which states are reachable from sq? 

b) Which states are reachable from sz? 

c) Which states are transient? 

d) Which states are sinks? 

Suppose that S, 7, and O are finite sets such that |S| = n, 

[Z| =k, and |O| =m. 

a) How many different finite-state machines (Mealy ma- 
chines) M = (S,1, O, f, g, 59) can be constructed, 
where the starting state sg can be arbitrarily chosen? 

b) How many different Moore machines M = 
(S, 1, O, f, g, 59) can be constructed, where the start- 
ing state sp can be arbitrarily chosen? 

Suppose that S$ and / are finite sets such that |$| =n 

and |J| =k. How many different finite-state automata 

M =(S, I, f, 50, F) are there where the starting state so 
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and the subset F of S consisting of final states can be c) with no three consecutive Os or two consecutive Is. 


chosen arbitrarily * 23. Show that if A is a regular set, then so is A. 
a) if the automata are deterministic? *24, Show that if A and B are regular sets, then so is AN B. 
b) if the automata may be nondeterministic? (Note: This *25, Find finite-state automata that recognize these sets of 


includes deterministic automata.) 
Construct a deterministic finite-state automaton that is 
equivalent to the nondeterministic automaton with the 
state diagram shown here. 


strings of Os and 1s. 
19 


a) the set of all strings that start with no more than three 
consecutive Os and contain at least two consecutive 1s 
b) the set of all strings with an even number of symbols 
that do not contain the pattern 101 
c) the set of all strings with at least three blocks of two 
or more Is and at least two Os 
*26. Show that {02" | n € N} is not regular. You may use the 
pumping lemma given in Exercise 22 of Section 13.4. 


. 


* 27. Show that {1” | p is prime} is not regular. You may use 


the pumping lemma given in Exercise 22 of Section 13.4. 


. 


* 28. There is a result for context-free languages analogous to 
the pumping lemma for regular sets. Suppose that L(G) 


is the language recognized by a context-free language G. 


. 


20. What is the language recognized by the automaton in Ex- 


ercise 19? 

Construct finite-state automata that recognize these sets. 
a) 0*(10)* 

b) (01 U 111)*10* (0 U1) 

c) (001 U (11)*)* 


This result states that there is a constant N such that if z 
is a word in L(G) with /(z) > N, then z can be written 
as uvwxy, where /(vwx) < N, (vx) > 1, and uviwxly 
belongs to L(G) fori = 0,1,2,3,.... Use this result 
to show that there is no context-free grammar G with 


#22 L(G) = {0"1"2" |n =0,1,2,...}. 


. 


Find regular expressions that represent the set of all strings 


of Os and Is * 29. Construct a Turing machine that computes the function 
a) made up of blocks of even numbers of 1s interspersed f (m1, 2) = max(ny, nz). 
with odd numbers of Os. * 30. Construct a Turing machine that computes the function 
b) with at least two consecutive Os or three consecutive f(,n2) =n2—n, if ng >ny, and f(n,,n2) = 0 if 
ls. n2 <n. 


Computer Projects 


Write programs with these input and output. 


1. Given the productions in a phrase-structure grammar, de- 7. Given the state table of a deterministic finite-state automa- 
termine which type of grammar this is in the Chomsky ton and a string, decide whether this string is recognized 
classification scheme. by the automaton. 

2. Given the productions of a phrase-structure grammar, find 8. Given the state table of a nondeterministic finite-state au- 


tomaton and a string, decide whether this string is recog- 


all strings that are generated using twenty or fewer appli- . 
g 8 g y PP nized by the automaton. 


cations of its production rules. ° ar . 
*9, Given the state table of a nondeterministic finite-state au- 


tomaton, construct the state table of a deterministic finite- 
state automaton that recognizes the same language. 


3. Given the Backus—Naur form of a type 2 grammar, find 
all strings that are generated using twenty or fewer appli- 


cations of the rules defining it. ** 10. Given a regular expression, construct a nondeterminis- 


*4, Given the productions of a context-free grammar and a tic finite-state automaton that recognizes the set that this 
string, produce a derivation tree for this string if it is in expression represents. 
the language generated by this grammar. 11. Given a regular grammar, construct a finite-state automa- 
; . . ton that recognizes the language generated by this gram- 
5. Given the state table of a Moore machine and an input mar. 
string, produce the output string generated by the ma- 12. Given a finite-state automaton, construct a regular gram- 
chine. i : : 
mar that generates the language recognized by this au- 
6. Given the state table of a Mealy machine and an input tomaton. 


string, produce the output string generated by the ma- * 13. Given a Turing machine, find the output string produced 
chine. by a given input string. 


. 
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Use a computational program or programs you have written to do these exercises. 


#2. 


. Solve the busy beaver problem for two states by test- 


ing all possible Turing machines with two states and 
alphabet {1, B}. 


Solve the busy beaver problem for three states by testing 
all possible Turing machines with three states and alphabet 
{1, B}. 


Writing Projects 


Respond to these with essays using outside sources. 


. Describe how the growth of certain types of plants can be 


modeled using a Lidenmeyer system. Such a system uses 
a grammar with productions modeling the different ways 
plants can grow. 


. Describe the Backus—Naur form (and extended Backus— 


Naur form) rules used to specify the syntax of a program- 
ming language, such as Java, LISP, or Ada, or the database 
language SQL. 


. Explain how finite-state machines are used by spell- 


checkers. 


. Explain how finite-state machines are used in the study 


of network protocols. 


. Explain how finite-state machines are used in speech 


recognition programs. 


. Compare the use of Moore machines versus Mealy ma- 


chines in the design of hardware systems and computer 
software. 


. Explain the concept of minimizing finite-state automata. 


Give an algorithm that carries out this minimization. 


. Give the definition of cellular automata. Explain their ap- 


plications. Use the Game of Life as an example. 


. Define a pushdown automaton. Explain how pushdown 


automata are used to recognize sets. Which sets are rec- 
ognized by pushdown automata? Provide an outline of a 
proof justifying your answer. 


** 3, Find a busy beaver machine with four states by testing 
all possible Turing machines with four states and alphabet 
{1, B}. 

** 4, Make as much progress as you can toward finding a busy 
beaver machine with five states. 

** 5, Make as much progress as you can toward finding a busy 
beaver machine with six states. 


10. Define a linear-bounded automaton. Explain how linear- 
bounded automata are used to recognize sets. Which sets 
are recognized by linear-bounded automata? Provide an 
outline of a proof justifying your answer. 


11. Look up Turing’s original definition of what we now call 
a Turing machine. What was his motivation for defining 
these machines? 


12. Describe the concept of the universal Turing machine. 
Explain how such a machine can be built. 


13. Explain the kinds of applications in which nondetermin- 
istic Turing machines are used instead of deterministic 
Turing machines. 


14. Show that a Turing machine can simulate any action of a 
nondeterministic Turing machine. 


15. Show that a set is recognized by a Turing machine if and 
only if it is generated by a phrase-structure grammar. 


16. Describe the basic concepts of the lambda-calculus and 
explain how it is used to study computability of functions. 


17. Show that a Turing machine as defined in this chapter can 
do anything a Turing machine with n tapes can do. 


18. Show that a Turing machine with a tape infinite in one 
direction can do anything a Turing machine with a tape 
infinite in both directions can do. 


Axioms for the Real Numbers 
and the Positive Integers 


L this book we have assumed an explicit set of axioms for the set of real numbers and for 
the set of positive integers. In this appendix we will list these axioms and we will illustrate 
how basic facts, also used without proof in the text, can be derived using them. 


Axioms for Real Numbers 


The standard axioms for real numbers include both the field (or algebraic) axioms, used to 
specify rules for basic arithmetic operations, and the order axioms, used to specify properties 
of the ordering of real numbers. 


THE FIELD AXIOMS We begin with the field axioms. As usual, we denote the sum and 
product of two real numbers x and y by x + y and x - y, respectively. (Note that the product of 
x and y is often denoted by xy without the use of the dot to indicate multiplication. We will not 
use this abridged notation in this appendix, but will within the text.) Also, by convention, we 
perform multiplications before additions unless parentheses are used. Although these statements 
are axioms, they are commonly called Jaws or rules. The first two of these axioms tell us that 
when we add or multiply two real numbers, the result is again a real number; these are the 
closure laws. 


= Closure law for addition For all real numbers x and y, x + y is a real number. 
= Closure law for multiplication For all real numbers x and y, x - y is a real number. 


The next two axioms tell us that when we add or multiply three real numbers, we get the 
same result regardless of the order of operations; these are the associative laws. 


® Associative law for addition For all real numbers x, y, and z, (n+ y)+z= 
x+(y +2). 
= Associative law for multiplication For all real numbers x, y, and z, (x-y)-z= 
x-(y-Z). 
Two additional algebraic axioms tell us that the order in which we add or multiply two 
numbers does not matter; these are the commutative laws. 


= Commutative law for addition For all real numbers x and y,x + y=y+x. 
= Commutative law for multiplication For all real numbers x and y,x- y= y-x. 


The next two axioms tell us that 0 and | are additive and multiplicative identities for the set 
of real numbers. That is, when we add 0 to a real number or multiply a real number by | we do 
not change this real number. These laws are called identity laws. 


= Additive identity law For every real number 
x,x+0=04+x=x. 
= Multiplicative identity law For every real number x,x-1=1-x =x. 


Although it seems obvious, we also need the following axiom. 


= Identity elements axiom The additive identity 0 and the multiplicative identity 1 are 
distinct, that is 0 A 1. 
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Two additional axioms tell us that for every real number, there is a real number that can be 
added to this number to produce 0, and for every nonzero real number, there is a real number 
by which it can be multiplied to produce |. These are the inverse laws. 


= Inverse law for addition For every real number «x, there exists a real number —x (called 
the additive inverse of x) such that x + (—x) = (-x) +x =0. 

= Inverse law for multiplication For every nonzero real number x, there exists a real 
number | /x (called the multiplicative inverse of x) such that x - (1/x) = (1/x)-x = 1. 


The final algebraic axioms for real numbers are the distributive laws, which tell us that 
multiplication distributes over addition; that is, that we obtain the same result when we first add 
a pair of real numbers and then multiply by a third real number or when we multiply each of 
these two real numbers by the third real number and then add the two products. 


= Distributive laws For all real numbers x, y, and z, x-(y+z)=x-y+x-z and 
(xty)-z=x-zZ+y-z. 


ORDER AXIOMS Next, we will state the order axioms for the real numbers, which specify 
properties of the “greater than” relation, denoted by >, on the set of real numbers. We write 
x > y (and y < x) when x is greater than y, and we write x > y (and y < x) when x > y 
or x = y. The first of these axioms tells us that given two real numbers, exactly one of three 
possibilities occurs: the two numbers are equal, the first is greater than the second, or the second 
is greater than the first. This rule is called the trichotomy law. 


= Trichotomy law For all real numbers x and y, exactly one of x = y,x > y,ory >x 
is true. 


Next, we have an axiom, called the transitivity law, that tells us that if one number is greater 
than a second number and this second number is greater than a third, then the first number is 
greater than the third. 


= Transitivity law For all real numbers x, y, and z, if x > y and y > z, then x > z. 


We also have two compatibility laws, which tell us that when we add a number to both sides 
in a greater than relationship, the greater than relationship is preserved and when we multiply 
both sides of a greater than relationship by a positive real number (that is, a real number x with 
x > 0), the greater than relationship is preserved. 


= Additive compatibility law For all real numbers x, y, and z, if x > y, then x +z > 
yz. 

= Multiplicative compatibility law For all real numbers x, y, and z,ifx > y andz > 0, 
thenx-z>y-z. 


We leave it to the reader (see Exercise 15) to prove that for all real numbers x, y, and z, if 
x > y and z <0, then x -z < y-z. That is, multiplication of an inequality by a negative real 
number reverses the direction of the inequality. 

The final axiom for the set of real numbers is the completeness property. Before we state 
this axiom, we need some definitions. First, given a nonempty set A of real numbers, we say 
that the real number b is an upper bound of A if for every real number a in A, b > a. A real 
number s is a least upper bound of A if s is an upper bound of A and whenever ¢ is an upper 
bound of A, then we have s < f. 


= Completeness property Every nonempty set of real numbers that is bounded above 
has a least upper bound. 
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Using Axioms to Prove Basic Facts 


The axioms we have listed can be used to prove many properties that are often used without 
explicit mention. We give several examples of results we can prove using axioms and leave the 
proof of a variety of other properties as exercises. Although the results we will prove seem quite 
obvious, proving them using only the axioms we have stated can be challenging. 


THEOREM 1 _ The additive identity element 0 of the real numbers is unique. 


Proof: To show that the additive identity element 0 of the real numbers is unique, suppose that 0’ 
is also an additive identity for the real numbers. This means that 0’ + x = x + 0! = x whenever 
x is a real number. By the additive identity law, it follows that 0 + 0’ = 0’. Because 0’ is an 
additive identity, we know that 0 + 0’ = 0. It follows that 0 = 0’, because both equal 0 + 0’. 
This shows that 0 is the unique additive identity for the real numbers. 4d 


THEOREM 2 _ The additive inverse of a real number x is unique. 


Proof: Let x be a real number. Suppose that y and z are both additive inverses of x. Then, 


y=O0O+y by the additive identity law 
=(z+x)+y because z is an additive inverse of x 


=z+(x+ y) _ by the associative law for addition 


=z+0 because y is an additive inverse of x 
=Z by the additive identity law. 
It follows that y = z. q 


Theorems | and 2 tell us that the additive identity and additive inverses are unique. Theo- 
rems 3 and 4 tell us that the multiplicative identity and multiplicative inverses of nonzero real 
numbers are also unique. We leave their proofs as exercises. 


THEOREM 3 _ The multiplicative identity element | of the real numbers is unique. 


THEOREM 4 _ The multiplicative inverse of a nonzero real number x is unique. 


THEOREM 5 For every real number x, x -0 = 0. 


Proof: Suppose that x is a real number. By the additive inverse law, there is a real number y that 
is the additive inverse of x - 0, so we have x -0 + y = 0. By the additive identity law, 0 + 0 = 0. 
Using the distributive law, we see that x -O0 = x-(0+0) =x -0+-.x - 0. It follows that 


O=x-0+y=(-04+x-0)4+y. 
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THEOREM 6 


THEOREM 7 


Next, note that by the associative law for addition and because x - 0+ y = 0, it follows that 
(x-0+%x-0)+y=x-04+(@-0+ y)=x-040. 


Finally, by the additive identity law, we know that x-0+0=.x.-0. Consequently, 
x-0=0. x 


For all real numbers x and y, if x - y = 0, thenx = Oory = 0. 


Proof: Suppose that x and y are real numbers and x - y = 0. Ifx ¥ 0, then, by the multiplicative 
inverse law, x has a multiplicative inverse 1/x, such that x - (1/x) = (1/x)-x = 1. Because 
x+y =0, we have (1/x)- (x - y) = (1/x) - 0 =0 by Theorem 5. Using the associate law for 
multiplication, we have ((1/x)-x)-y = 0. This means that | - y = 0. By the multiplicative 
identity rule, we see that | - y = y, so y = 0. Consequently, either x = 0 or y = 0. | 


The multiplicative identity element | in the set of real numbers is greater than the additive 
identity element 0. 


Proof: By the trichotomy law, either 0 = 1,0 > 1, or 1 > 0. We know by the identity elements 
axiom that 0 4 1. 

So, assume that 0 > 1. We will show that this assumption leads to a contradiction. 
By the additive inverse law, 1 has an additive inverse —1 with 1+ (—1) =0. The addi- 
tive compatibility law tells us that 0 + (—1) > 1+ (—1) = 0; the additive identity law tells 
us that 0 + (—1) = —1. Consequently, —1 > 0, and by the multiplicative compatibility law, 
(—1)-(—1) > (-1) - 0. By Theorem 5 the right-hand side of last inequality is 0. By the dis- 
tributive law, (—1) - (—1) + (-1)- 1 = (-1)- (-14+ 1) = (-1)- 0 = 0. Hence, the left-hand 
side of this last inequality, (—1) - (—1), is the unique additive inverse of —1, so this side of 
the inequality equals 1. Consequently this last inequality becomes | > 0, contradicting the 
trichotomy law because we had assumed that 0 > 1. 

Because we know that 0 4 | and that it is impossible for 0 > 1, by the trichotomy law, we 
conclude that 1 > 0. | 


ARCHIMEDES (287 B.c.E.-212 B.c.E.) Archimedes was one of the greatest scientists and mathematicians of 
ancient times. He was born in Syracuse, a Greek city-state in Sicily. His father, Phidias, was an astronomer. 
Archimedes was educated in Alexandria, Egypt. After completing his studies, he returned to Syracuse, where 
he spent the rest of his life. Little is known about his personal life; we do not know whether he was ever married 
or had children. Archimedes was killed in 212 B.c.E. by a Roman soldier when the Romans overran Syracuse. 
Archimedes made many important discoveries in geometry. His method for computing the area under a 
curve was described two thousand years before his ideas were re-invented as part of integral calculus. Archimedes 
also developed a method for expressing large integers inexpressible by the usual Greek method. He discov- 
ered a method for computing the volume of a sphere, as well as of other solids, and he calculated an ap- 
proximation of 2. Archimedes was also an accomplished engineer and inventor; his machine for pumping 
water, now called Archimedes’ screw, is still in use today. Perhaps his best known discovery is the principle of 


buoyancy, which tells us that an object submerged in liquid becomes lighter by an amount equal to the weight it displaces. Some 
histories tell us that Archimedes was an early streaker, running naked through the streets of Syracuse shouting “Eureka” (which 
means “I have found it”) when he made this discovery. He is also known for his clever use of machines that held off Roman forces 
sieging Syracuse for several years during the Second Punic War. 


THEOREM 8 
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The next theorem tells us that for every real number there is an integer (where by an integer, 
we mean 0, the sum of any number of Is, and the additive inverses of these sums) greater than 
this real number. This result is attributed to the Greek mathematician Archimedes. The result 
can be found in Book V of Euclid’s Elements. 


ARCHIMEDEAN PROPERTY For every real number x there exists an integer n such 
thatn > x. 


Proof: Suppose that x is a real number such that n < x for every integer n. Then x is an upper 
bound of the set of integers. By the completeness property it follows that the set of integers has a 
least upper bound M. Because M — | < M and M isa least upper bound of the set of integers, 
M — 1 is not an upper bound of the set of integers. This means that there is an integer n with 
n > M — 1. This implies that n + 1 > M, contradicting the fact that M is an upper bound of 
the set of integers. 


Axioms for the Set of Positive Integers 


The axioms we now list specify the set of positive integers as 
the subset of the set of integers satisfying four key properties. We assume the truth of these 
axioms in this textbook. 


= Axiom 1 = The number | is a positive integer. 

= Axiom 2 If 7 is a positive integer, then n + 1, the successor of n, is also a positive 
integer. 

= Axiom3 _ Every positive integer other than | is the successor of a positive integer. 


= Axiom 4 The Well-Ordering Property Every nonempty subset of the set of positive 
integers has a least element. 


In Sections 5.1 and 5.2 it is shown that the well-ordering principle is equivalent to the 
principle of mathematical induction. 


= Mathematical induction axiom If S is a set of positive integers such that | € S and 
for all positive integers n ifn € S, thenn + 1 € S, then S is the set of positive integers. 


Most mathematicians take the real number system as already existing, with the real numbers 
satisfying the axioms we have listed in this appendix. However, mathematicians in the nineteenth 
century developed techniques to construct the set of real numbers, starting with more basic sets 
of numbers. (The process of constructing the real numbers is sometimes studied in advanced 
undergraduate mathematics classes. A treatment of this can be found in [Mo91], for instance.) 
The first step in the process is the construction of the set of positive integers using axioms 1-3 
and either the well-ordering property or the mathematical induction axiom. Then, the operations 
of addition and multiplication of positive integers are defined. Once this has been done, the set 
of integers can be constructed using equivalence classes of pairs of positive integers where 
(a, b) ~ (c,d) if and only if a+ d=b-+c; addition and multiplication of integers can be 
defined using these pairs (see Exercise 21). (Equivalence relations and equivalence classes 
are discussed in Chapter 9.) Next, the set of rational numbers can be constructed using the 
equivalence classes of pairs of integers where the second integer in the pair is not zero, where 
(a, b) © (c, d) if and only if a. d = b - c; addition and multiplication of rational numbers can 
be defined in terms of these pairs (see Exercise 22). Using infinite sequences, the set of real 
numbers can then be constructed from the set of rational numbers. The interested reader will 
find it worthwhile to read through the many details of the steps of this construction. 
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Exercises 


Use only the axioms and theorems in this appendix in the 
proofs in your answers to these exercises. 


1. 


2. 


6. 


Ts 


Prove Theorem 3, which states that the multiplicative 
identity element of the real numbers is unique. 

Prove Theorem 4, which states that for every nonzero real 
number x, the multiplicative inverse of x is unique. 


. Prove that for all real numbers x and y, (—x)-y= 


x-(-y) =—(- y). 


. Prove that for all real numbers x and y, —(x+ y) = 
(—x) + (-y). 

. Prove that for all real numbers x and y, (—x)-(—y) = 
x-y. 
Prove that for all real numbers x, y, and z, if x +z= 


y+z, thenx = y. 
Prove that for every real number x, —(—x) = x. 


Define the difference x — y of real numbers x and y by 
x —y=x+(-—y), where —y is the additive inverse of y, 
and the quotient x/y, where y 40, by x/y =x - (1/y), 
where 1/y is the multiplicative inverse of y. 


8. 


9. 


10. 


11. 


12. 


13. 


Prove that for all real numbers x and y, x = y if and only 
ifx—y=0. 

Prove that for all real numbers x and y, —x — y = —(x + 
y). 

Prove that for all nonzero real numbers x and y, 
1/(x/y) = y/x, where 1/(x/y) is the multiplicative in- 
verse of x/y. 

Prove that for all real numbers w, x, y, and z, if x #0 
and z # 0, then (w/x) + (y/z) = (w-z+x- y)/(x +z). 
Prove that for every positive real number x, 1/x is alsoa 
positive real number. 

Prove that for all positive real numbers x and y, x - y is 
also a positive real number. 


14. 


15. 


16. 


17. 


18. 


19. 


* 20. 


Prove that for all real numbers x and y, if x > 0 and 

y <0,thenx-y <0. 

Prove that for all real numbers x, y, and z, if x > y and 

z<0O,thenx-z<y-z. 

Prove that for every real number x, x ~ 0 if and only if 
2 

x“ >0. 

Prove that for all real numbers w, x, y, and z, if w <x 

and y < z,thenw+y<x+z. 

Prove that for all positive real numbers x and y, if x < y, 

then 1/x > 1/y. 


Prove that for every positive real number x, there exists 
a positive integer n such thatn-x > 1. 


Prove that between every two distinct real numbers there 


is a rational number (that is, a number of the form x/y, 
where x and y are integers with y # 0). 


Exercises 21 and 22 involve the notion of an equivalence re- 
lation, discussed in Chapter 9 of the text. 


*21. 


* 22. 


Define a relation ~ on the set of ordered pairs of pos- 
itive integers by (w, x) ~ (y, z) if and only if w+ z= 
x + y. Show that the operations [(w, x)]~ +[Q, z)]~ = 
[w+y, x42] and [(w, ole 10,2]. =[w- y+ 
x-+Z,x-y-+w- z)]~ are well-defined, that is, they do not 
depend on the representative of the equivalence classes 
chosen for the computation. 


Define a relation © on ordered pairs of integers with sec- 
ondentry nonzero by (w, x) © (y, z) ifandonlyifw-z = 
x - y. Show that the operations [(w, x)]~ +[0y, Jl~ = 
[wz +x-y,x-z)]~ and [(w, x)]~ -[, De = Lowry, 
x -Z)]~ are well-defined, that is, they do not depend on 
the representative of the equivalence classes chosen for 
the computation. 


THEOREM 1 


Functions 


i this appendix we review some of the basic properties of exponential functions and log- 
arithms. These properties are used throughout the text. Students requiring further review 
of this material should consult precalculus or calculus books, such as those mentioned in the 
Suggested Readings. 


Exponential Functions 


Let n be a positive integer, and let b be a fixed positive real number. The function f,(n) = b” 
is defined by 


foln) = BY = b+ bebe + +b, 


where there are n factors of b multiplied together on the right-hand side of the equation. 

We can define the function f(x) = b* for all real numbers x using techniques from calculus. 
The function f,(x) = b* is called the exponential function to the base b. We will not discuss 
how to find the values of exponential functions to the base b when x is not an integer. 

Two of the important properties satisfied by exponential functions are given in Theorem 1. 
Proofs of these and other related properties can be found in calculus texts. 


Let b be a positive real number and x and y real numbers. Then 
6 — pb and 
Ds OY Sa. 


We display the graphs of some exponential functions in Figure 1. 


Logarithmic Functions 


Suppose that b is areal number with b > 1. Then the exponential function b* is strictly increasing 
(a fact shown in calculus). It is a one-to-one correspondence from the set of real numbers to the set 
of nonnegative real numbers. Hence, this function has an inverse log, x, called the logarithmic 
function to the base b. In other words, if b is a real number greater than | and x is a positive 
real number, then 


pl8o* — x, 


The value of this function at x is called the logarithm of x to the base b. 
From the definition, it follows that 


log, b* =x. 


We give several important properties of logarithms in Theorem 2. 
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FIGURE 1 Graphs of the Exponential Functions to the Bases ay 2, and 5. 


THEOREM 2 Let b be areal number greater than 1. Then 


1. log, (xy) = log, x + log, y whenever x and y are positive real numbers, and 
2. log, (x) = y log, x whenever x is a positive real number and y is a real number. 


Proof: Because log, (xy) is the unique real number with b!°&)°') — xy, to prove part | it suffices 
to show that b's *+!08, ¥ — xy. By part 1 of Theorem 1, we have 


plog, x+log, y — plog, x plog, y 


=xXy. 
To prove part 2, it suffices to show that bY !°8o* = x”. By part 2 of Theorem 1, we have 


bY logy * = (p!8*yy 
= x. 4 


The following theorem relates logarithms to two different bases. 


THEOREM 3 Leta and bd be real numbers greater than 1, and let x be a positive real number. Then 


log, x = log, x/log, a. 


Proof: To prove this result, it suffices to show that 


p!8a x-log,a _ x. 


1 2 3 


FIGURE 2 The Graph of f(x) = log x. 


By part 2 of Theorem 1, we have 


p'08a ¥ log, a — (pS a)108q x 
= q!8aX 


=X. 


This completes the proof. 
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Because the base used most often for logarithms in this text is b = 2, the notation log x is 


used throughout the test to denote log, x. 


The graph of the function f(x) = log x is displayed in Figure 2. From Theorem 3, when a 
base b other than 2 is used, a function that is a constant multiple of the function log x, namely, 


(1/ log b) log x, is obtained. 


Exercises 


. Express each of the following quantities as powers of 2. 


a) 2-22 b) (22)3 ce) 22 
. Find each of the following quantities. 
a) log, 1024 b) log, 1/4 c) log, 8 


. Suppose that logy x = y where x is a positive real number. 
Find each of the following quantities. 


a) log, x b) log x c) logigx 


c= 4, 


5. 


Let a, b, and c be positive real numbers. Show that q8pe = 
log, a 
crepe, 


Draw the graph of f(x) = b’ for all real numbers x if b 
is 


a) 3. b) 1/3. c) 1. 

. Draw the graph of f(x) = log, x for positive real numbers 
x if bis 
a) 4. b) 100. c) 1000. 


Links 


Pseudocode 


T he algorithms in this text are described both in English and in pseudocode. Pseudocode is 
an intermediate step between an English language description of the steps of a procedure 
and a specification of this procedure using an actual programming language. The advantages 
of using pseudocode include the simplicity with which it can be written and understood and 
the ease of producing actual computer code (in a variety of programming languages) from the 
pseudocode. We will describe the particular types of statements, or high-level instructions, of 
the pseudocode that we will use. Each of these statements in pseudocode can be translated into 
one or more statements in a particular programming language, which in turn can be translated 
into one or more (possibly many) low-level instructions for a computer. 

This appendix describes the format and syntax of the pseudocode used in the text. This 
pseudocode is designed so that its basic structure resembles that of commonly used programming 
languages, such as C++ and Java, which are currently the most commonly taught programming 
languages. However, the pseudocode we use will be a lot looser than a formal programming 
language because a lot of English language descriptions of steps will be allowed. 

This appendix is not meant for formal study. Rather, it should serve as a reference guide for 
students when they study the descriptions of algorithms given in the text and when they write 
pseudocode solutions to exercises. 


Procedure Statements 


The pseudocode for an algorithm begins with a procedure statement that gives the name of 
an algorithm, lists the input variables, and describes what kind of variable each input is. For 
instance, the statement 


procedure maximum(L: list of integers) 


is the first statement in the pseudocode description of the algorithm, which we have named 
maximum, that finds the maximum of a list L of integers. 


Assignments and Other Types of Statements 


An assignment statement is used to assign values to variables. In an assignment statement the 
left-hand side is the name of the variable and the right-hand side is an expression that involves 
constants, variables that have been assigned values, or functions defined by procedures. The 
right-hand side may contain any of the usual arithmetic operations. However, in the pseudocode 
in this book it may include any well-defined operation, even if this operation can be carried out 
only by using a large number of statements in an actual programming language. 

The symbol := is used for assignments. Thus, an assignment statement has the form 


variable := expression 
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For example, the statement 


Max := a 


assigns the value of a to the variable max. A statement such as 


x := largest integer in the list L 


can also be used. This sets x equal to the largest integer in the list L. To translate this statement into 
an actual programming language would require more than one statement. Also, the instruction 


interchange a and b 


can be used to interchange a and b. We could also express this one statement with several 
assignment statements (see Exercise 2), but for simplicity, we will often prefer this abbreviated 
form of pseudocode. 


Comments 


In the pseudocode in this book, statements enclosed in curly braces are not executed. Such 
statements serve as comments or reminders that help explain how the procedure works. For 
instance, the statement 


{x is the largest element in L} 


can be used to remind the reader that at that point in the procedure the variable x equals the 
largest element in the list L. 


Conditional Constructions 


The simplest form of the conditional construction that we will use is 


if condition then statement 


or 


if condition then 
block of statements 
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Here, the condition is checked, and if it is true, then the statement or block of statements given 
is carried out. In particular, the pseudocode 


if condition then 
statement 1 
statement 2 
statement 3 


statement n 


tells us that the statements in the block are executed sequentially if the condition is true. 

For example, in Algorithm | in Section 3.1, which finds the maximum of a set of integers, 
we use a conditional statement to check whether max < a; for each variable; if it is, we assign 
the value of a; to max. 

Often, we require the use of a more general type of construction. This is used when we wish 
to do one thing when the indicated condition is true, but another when it is false. We use the 
construction 


if condition then statement | 
else statement 2 


Note that either one or both of statement | and statement 2 can be replaced with a block of 
statements. 

Sometimes, we require the use of an even more general form of a conditional. The general 
form of the conditional construction that we will use is 


if condition 1 then statement 1 
else if condition 2 then statement 2 
else if condition 3 then statement 3 


else if condition n then statement n 
else statement n + 1 


When this construction is used, if condition | is true, then statement 1 is carried out, and the 
program exits this construction. In addition, if condition | is false, the program checks whether 
condition 2 is true; if it is, statement 2 is carried out, and so on. Thus, if none of the first n — 1 
conditions hold, but condition n does, statement n is carried out. Finally, if none of condition 
1, condition 2, condition 3, ..., condition 7 is true, then statement n + | is executed. Note that 
any of the n + | statements can be replaced by a block of statements. 
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Loop Constructions 


There are two types of loop construction in the pseudocode in this book. The first is the “for” 
construction, which has the form 


for variable := initial value to final value 
statement 


or 


for variable := initial value to final value 
block of statements 


where initial value and final value are integers. Here, at the start of the loop, variable is assigned 
initial value if initial value is less than or equal to final value, and the statements at the end of 
this construction are carried out with this value of variable. Then variable is increased by one, 
and the statement, or the statements in the block, are carried out with this new value of variable. 
This is repeated until variable reaches final value. After the instructions are carried out with 
variable equal to final value, the algorithm proceeds to the next statement. When initial value 
exceeds final value, none of the statements in the loop is executed. 

We can use the “for” loop construction to find the sum of the positive integers from | ton 
with the following pseudocode. 


sum :=0 
fori :=1ton 
sum := sum +1 


Also, the more general “for” statement, of the form 


for all elements with a certain property 


is used in this text. This means that the statement or block of statements that follow are carried 
out successively for the elements with the given property. 

The second type of loop construction that we will use is the “while” construction. This has 
the form 


while condition 
statement 


or 


while condition 
block of statements 


When this construction is used, the condition given is checked, and if it is true, the statements that 
follow are carried out, which may change the values of the variables that are part of the condition. 
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If the condition is still true after these instructions have been carried out, the instructions are 
carried out again. This is repeated until the condition becomes false. As an example, we can 
find the sum of the integers from | to 1 using the following block of pseudocode including a 
“while” construction. 


Suna 

while n > 0 
sum := sum +n 
ni=n-1 


Note that any “for” construction can be turned into a “while” construction (see Exercise 3). 
However, it is often easier to understand the “for” construction. So, when it makes sense, we 
will use the “for” construction in preference to the corresponding “while” construction. 


Loops within Loops 


Loops or conditional statements are often used within other loops or conditional statements. 
In the pseudocode used in this book, we use successive levels of indentation to indicate nested 
loops, which are loops within loops, and which blocks of commands correspond to which loops. 


Using Procedures in Other Procedures 


We can use a procedure from within another procedure (or within itself in a recursive program) 
simply by writing the name of this procedure followed by the inputs to this procedure. For 
instance, 


max(L) 


will carry out the procedure max with the input list L. After all the steps of this procedure have 
been carried out, execution carries on with the next statement in the procedure. 


Return Statements 


We use a return statement to show where a procedure produces output. A return statement of 
the form 


return x 


produces the current value of x as output. The output x can involve the value of one or more 
functions, including the same function under evaluation, but at a smaller value. For instance, 
the statement 


return f(n — 1) 


is used to call the algorithm with input of n — 1. This means that the algorithm is run again with 
input equal ton — 1. 
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Exercises 
1. What is the difference between the following blocks of two 2. Give a procedure using assignment statements to inter- 
assignment statements? change the values of the variables x and y. What is the 
minimum number of assignment statements needed to do 
a:=b this? 
b:=c 3. Show how a loop of the form 
and for i := initial value to final value 
statement 
b:=c 


can be written using the “while” construction. 


Suggested Reading 


A mong the resources available for learning more about the topics covered in this book are printed materials 
and relevant websites. Printed resources are described in this section of suggested readings. Readings 
are listed by chapter and keyed to particular topics of interest. Some general references also deserve special 
mention. A book you may find particularly useful is the Handbook of Discrete and Combinatorial Mathematics 
by Rosen [Ro00], acomprehensive reference book. Additional applications of discrete mathematics can be found 
in Michaels and Rosen [MiRo91], which is also available online on the companion website for this book. Deeper 
coverage of many topics in computer science, including those discussed in this book, can be found in Gruska 
[Gr97]. Biographical information about many of the mathematicians and computer scientists mentioned in this 
book can be found in Gillispie [Gi70] and on the MacTutor website at /itip:/Avww-history.mcs.st-and.ac.uk/. 


To find pertinent websites, consult the links found in the Web Resources Guide on the companion website 
for this book. Its address is www.mhhe.com/rosen. 


CHAPTER 1 


An entertaining way to study logic is to read Lewis Carroll’s book 
[Ca78]. General references for logic include M. Huth and M. Ryan 
[HuRy04], Mendelson [Me09], Stoll [St74], and Suppes [Su87]. A 
comprehensive treatment of logic in discrete mathematics can be 
found in Gries and Schneider [GrSc93]. System specifications are 
discussed in Ince [In93]. Smullyan’s knights and knaves puzzles 
were introduced in [Sm78]. He has written many fascinating books 
on logic puzzles including [Sm92] and [Sm98]. Prolog is discussed 
in depth in Nilsson and Maluszynski [NiMa95] and in Clocksin 
and Mellish [ClIMe94]. The basics of proofs are covered in Cu- 
pillari [Cu05], Morash [Mo91], Solow [S009], Velleman [Ve06], 
and Wolf [Wo98]. The science and art of constructing proofs is 
discussed in a delightful way in three books by Pélya: [Po62], 
[Po71], and [Po90]. Problems involving tiling checkerboards us- 
ing dominoes and polyominoes are discussed in Golomb [Go94] 
and Martin [Ma91]. 


CHAPTER 2 


Lin and Lin [LiLi81] is an easily read text on sets and their applica- 
tions. Axiomatic developments of set theory can be found in Hal- 
mos [Ha60], Monk [Mo69], and Stoll [St74]. Brualdi [Br09], and 
Reingold, Nievergelt, and Deo [ReNiDe77] contain introductions 
to multisets. Fuzzy sets and their application to expert systems 
and artificial intelligence are treated in Negoita [Ne85] and Zim- 
merman [Zi91]. Calculus books, such as Apostol [Ap67], Spivak 
[Sp94], and Thomas and Finney [ThFi96], contain discussions 
of functions. The best printed source of information about inte- 
ger sequences is Sloan and Plouffe [SIP195]. Books on proofs, 
such as [Ve06], often cover countability in some depth. Stanat and 
McAllister [StMc77] has a thorough section on countability. Chap- 
ter 17 of Aigner, Ziegler, and Hoffman [AiZiHo09] provides an 
excellent discussion of cardinality and the continuum hypothesis. 


Discussions of the mathematical foundations needed for computer 
science can be found in Arbib, Kfoury, and Moll [ArKfMo80], Bo- 
brow and Arbib [BoAr74], Beckman [Be80], and Tremblay and 
Manohar [TrMa75]. Matrices and their operations are covered in 
all linear algebra books, such as Curtis [Cu84] and Strang [St09]. 


CHAPTER 3 


The articles by Knuth [Kn77] and Wirth [Wi84] are accessible 
introductions to the subject of algorithms. Among the best intro- 
ductions to algorithms are Cormen, Leierson, Rivest, and Stein 
[CoLeRiSt09] and Kleinberg and Tardos [KITa05]. Extensive ma- 
terial on big-O estimates of functions can be found in Knuth 
[Kn97a]. General references for algorithms and their complex- 
ity include Aho, Hopcroft, and Ullman [AhHoUI74]; Baase and 
Van Gelder [BaGe99]; Cormen, Leierson, Rivest, and Stein [Co- 
LeRiSt09]; Gonnet [Go84]; Goodman and Hedetniemi [GoHe77]; 
Harel [Ha87]; Horowitz and Sahni [HoSa82]; Kreher and Stin- 
son [KrSt98]; the famous series of books by Knuth on the art of 
computer programming [Kn97a], [Kn97b], and [Kn98]; Kronsjé 
[Kr87]; Levitin [Le06]; Manber [Ma89]; Pohl and Shaw [PoSh8 1]; 
Purdom and Brown [PuBr85]; Rawlins [Ra92]; Sedgewick [Se03]; 
Wilf [Wi02]; and Wirth [Wi76]. Sorting and searching algorithms 
and their complexity are studied in detail in Knuth [Kn98]. 


CHAPTER 4 


References for number theory include Hardy and Wright [HaWr- 
WiHe08]; Le Veque [Le77]; Rosen [Ro10]; and Stark [St78]. More 
about the history of number theory can be found in Ore [Or88]. Al- 
gorithms for computer arithmetic are discussed in Knuth [Kn97b] 
and Pohl and Shaw [PoSh81]. More information about algorithms 
for finding primes and for factorization can be found in Crandall 
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and Pomerance [CrPo10]. Applications of number theory to cryp- 
tography are covered in Denning [De82]; Menezes, van Oorschot, 
and Vanstone [MeOoVa97]; Rosen [Ro10]; Seberry and Pieprzyk 
[SePi89]; Sinkov [Si66]; and Stinson [St05]. The RSA public- 
key system was described by Rivest, Shamir, and Adleman in 
[RiShAd78]; its discovery by Cocks is described in [Si99], which 
also provides an appealing account of the history of cryptography. 


CHAPTER 5 


Anaccessible introduction to mathematical induction can be found 
[Gu10] and in Sominskii [So61]. Books that contain thorough 
treatments of mathematical induction and recursive definitions in- 
clude Liu [Li85]; Sahni [Sa85]; Stanat and McAllister [StMc77]; 
and Tremblay and Manohar [TrMa75]. Computational geometry 
is covered in [DeOr11] and [Or00]. The Ackermann function, in- 
troduced in 1928 by W. Ackermann, arises in the theory of re- 
cursive function (see Beckman [Be80] and McNaughton [Mc82], 
for instance) and in the analysis of the complexity of certain set 
theoretic algorithms (see Tarjan [Ta83]). Recursion is studied in 
Roberts [Ro86]; Rohl [Ro84]; and Wand [Wa80]. Discussions of 
program correctness and the logical machinery used to prove that 
programs are correct can be found in Alagic and Arbib [AIAr78]; 
Anderson [An79]; Backhouse [Ba86]; Sahni [Sa85]; and Stanat 
and McAllister [StMc77]. 


CHAPTER 6 


General references for counting techniques and their applications 
include Allenby and Slomson [AIS110]; Anderson [An89]; Berman 
and Fryer [BeFr72]; Bogart [Bo00]; Bona [Bo07]; Bose and Man- 
vel [BoMa86]; Brualdi [Br09]; Cohen [Co78]; Grimaldi [Gr03]; 
Gross [Gr07]; Liu [Li68]; Polya, Tarjan, and Woods [PoTaWo83]; 
Riordan [Ri58]; Roberts and Tesman [RoTe03]; Tucker [Tu06]; 
and Williamson [Wi85]. Vilenkin [Vi71] contains a selection of 
combinatorial problems and their solutions. A selection of more 
difficult combinatorial problems can be found in Lovasz [L079]. 
Information about Internet protocol addresses and datagrams can 
be found in Comer [Co05]. Applications of the pigeonhole prin- 
ciple can be found in Brualdi [Br09]; Liu [Li85]; and Roberts and 
Tesman [RoTe03]. A wide selection of combinatorial identities can 
be found in Riordan [Ri68] and in Benjamin and Quinn [BeQu03]. 
Combinatorial algorithms, including algorithms for generating 
permutations and combinations, are described by Even [Ev73]; 
Lehmer [Le64]; and Reingold, Nievergelt, and Deo [ReNiDe77]. 


CHAPTER 7 


Useful references for discrete probability theory include Feller 
[Fe68], Nabin [Na00], and Ross [Ro09a]. Ross [Ro02], which fo- 
cuses on the application of probability theory to computer science, 
provides examples of average case complexity analysis and cov- 
ers the probabilistic method. Aho and Ullman [AhUI95] includes 
a discussion of various aspects of probability theory important in 


computer science, including programming applications of proba- 
bility. The probabilistic method is discussed in a chapter in Aigner, 
Ziegler, and Hoffman [AiZiHo009], a monograph devoted to clever, 
insightful, and brilliant proofs, that is, proofs that Paul Erdés de- 
scribed as coming from “The Book.” Extensive coverage of the 
probabilistic method can be found in Alon and Spencer [AISp00]. 
Bayes’ theorem is covered in [PaPi01]. Additional material on 
spam filters can be found in [Zd05]. 


CHAPTER 8 


Many different models using recurrence relations can be found 
in Roberts and Tesman [RoTe03] and Tucker [Tu06]. Exhaus- 
tive treatments of linear homogeneous recurrence relations with 
constant coefficients, and related inhomogeneous recurrence re- 
lations, can be found in Brualdi [Br09], Liu [Li68], and Mattson 
[Ma93]. Divide-and-conquer algorithms and their complexity are 
covered in Roberts and Tesman [RoTe03] and Stanat and McAI- 
lister [StMc77]. Descriptions of fast multiplication of integers and 
matrices can be found in Aho, Hopcroft, and Ullman [AhHoUI74] 
and Knuth [Kn97b]. An excellent introduction to generating func- 
tions can be found in Polya, Tarjan, and Woods [PoTaWo83]. Gen- 
erating functions are studied in detail in Brualdi [Br09]; Cohen 
[Co78]; Graham, Knuth, and Patashnik [GrknPa94]; Grimaldi 
[Gr03]; and Roberts and Tesman [RoTe03]. Additional applica- 
tions of the principle of inclusion—exclusion can be found in Liu 
[Li85] and [Li68]; Roberts and Tesman [RoTe03]; and Ryser 
[Ry63]. 


CHAPTER 9 


General references for relations, including treatments of equiv- 
alence relations and partial orders, include Bobrow and Ar- 
bib [BoAr74]; Grimaldi [Gr03]; Sanhi [Sa85]; and Tremblay 
and Manohar [TrMa75]. Discussions of relational models for 
databases are given in Date [Da82] and Aho and Ullman [AhUI195]. 
The original papers by Roy and Warshall for finding transitive clo- 
sures can be found in [Ro59] and [Wa62], respectively. Directed 
graphs are studied in Chartrand, Lesniak, and Zhang [ChLeZh05]; 
Gross and Yellen [GrYe05]; Robinson and Foulds [RoFo80]; 
Roberts and Tesman [RoTe03]; and Tucker [Tu06]. The applica- 
tion of lattices to information flow is treated in Denning [De82]. 


CHAPTER 10 


General references for graph theory include Agnarsson and 
Greenlaw [AgGr06]; Aldous, Wilson, and Best [AlWiBe00]; Be- 
hzad and Chartrand [BeCh71]; Chartrand, Lesniak, and Zhang 
[ChLeZh05]; Chartrand and Zhang [ChZh04]; Bondy and Murty 
[BoMul10]; Chartrand and Oellermann [ChOe93]; Graver and 
Watkins [GrWa77]; Roberts and Tesman [RoTe03]; Tucker 
[Tu06]; West [We00]; Wilson [Wi85]; and Wilson and Watkins 
[WiWa90]. A wide variety of applications of graph theory can be 
found in Chartrand [Ch77]; Deo [De74]; Foulds [Fo92]; Roberts 


and Tesman [RoTe03]; Roberts [Ro76]; Wilson and Beineke 
[WiBe79]; and McHugh [Mc90]. In depth treatments of the use of 
graph theory to study social networks, and other types of networks, 
appears in Easley and Kleinberg [EaK110] and Newman [Ne10]. 
Applications involving large graphs, including the Web graph, are 
discussed in Hayes [Ha00a] and [Ha00b]. 

A comprehensive description of algorithms in graph the- 
ory can be found in Gibbons [Gi85] and in Kocay and Kre- 
her [KoKr04]. Other references for algorithms in graph theory 
include Buckley and Harary [BuHa90]; Chartrand and Oeller- 
mann [ChOe93]; Chachra, Ghare, and Moore [ChGhMo79]; Even 
[Ev73] and [Ev79]; Hu [Hu82]; and Reingold, Nievergelt, and 
Deo [ReNiDe77]. A translation of Euler’s original paper on the 
KGnigsberg bridge problem can be found in Euler [Eu53]. Di- 
jkstra’s algorithm is studied in Gibbons [Gi85]; Liu [Li85]; and 
Reingold, Nievergelt, and Deo [ReNiDe77]. Dijkstra’s original 
paper can be found in [Di59]. A proof of Kuratowski’s theorem 
can be found in Harary [Ha69] and Liu [Li68]. Crossing num- 
bers and thicknesses of graphs are studied in Chartrand, Lesniak, 
and Zhang [ChLeZh05]. References for graph coloring and the 
four-color theorem are included in Barnette [Ba83] and Saaty and 
Kainen [SaKa86]. The original conquest of the four-color theo- 
rem is reported in Appel and Haken [ApHa76]. Applications of 
graph coloring are described by Roberts and Tesman [RoTe03]. 
The history of graph theory is covered in Biggs, Lloyd, and Wilson 
[BiL]Wi86]. Interconnection networks for parallel processing are 
discussed in Akl [Ak89] and Siegel and Hsu [SiHs88]. 


CHAPTER 11 


Trees are studied in Deo [De74], Grimaldi [Gr03], Knuth [Kn97a], 
Roberts and Tesman [RoTe03], and Tucker [Tu06]. The use of 
trees in computer science is described by Gotlieb and Gotlieb 
[GoGo78], Horowitz and Sahni [HoSa82], and Knuth [Kn97a, 
98]. Roberts and Tesman [RoTe03] covers applications of trees to 
many different areas. Prefix codes and Huffman coding are cov- 
ered in Hamming [Ha80]. Backtracking is an old technique; its 
use to solve maze puzzles can be found in the 1891 book by Lu- 
cas [Lu91]. An extensive discussion of how to solve problems 
using backtracking can be found in Reingold, Nievergelt, and Deo 
[ReNiDe77]. Gibbons [Gi85] and Reingold, Nievergelt, and 
Deo [ReNiDe77] contain discussions of algorithms for construct- 
ing spanning trees and minimal spanning trees. The background 
and history of algorithms for finding minimal spanning trees is cov- 
ered in Graham and Hell [GrHe85]. Prim and Kruskal described 
their algorithms for finding minimal spanning trees in [Pr57] and 
[Kr56], respectively. Sollin’s algorithm is an example of an al- 
gorithm well suited for parallel processing; although Sollin never 
published a description of it, his algorithm has been described by 
Even [Ev73] and Goodman and Hedetniemi [GoHe77]. 


CHAPTER 12 


Boolean algebra is studied in Hohn [Ho66], Kohavi [Ko86], and 
Tremblay and Manohar [TrMa75]. Applications of Boolean al- 
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gebra to logic circuits and switching circuits are described by 
Hayes [Ha93], Hohn [Ho66], Katz and Borriello [KaBo04], and 
Kohavi [Ko86]. The original papers dealing with the minimization 
of sum-of-products expansions using maps are Karnaugh [Ka53] 
and Veitch [Ve52]. The Quine-McCluskey method was introduced 
in McCluskey [Mc56] and Quine [Qu52] and [Qu55]. Threshold 
functions are covered in Kohavi [Ko86]. 


CHAPTER 13 


General references for formal grammars, automata theory, and 
the theory of computation include Davis, Sigal, and Weyuker 
[DaSiWe94]; Denning, Dennis, and Qualitz [DeDeQu81]; 
Hopcroft, Motwani, and Ullman [HoMoUI106]; Hopkin and Moss 
[HoMo76]; Lewis and Papadimitriou [LePa97]; McNaughton 
[Mc82]; and Sipser [Si06]. Mealy machines and Moore machines 
were originally introduced in Mealy [Me55] and Moore [Mo56]. 
The original proof of Kleene’s theorem can be found in [K156]. 
Powerful models of computation, including pushdown automata 
and Turing machines, are discussed in Brookshear [Br89], Hen- 
nie [He77], Hopcroft and Ullman [HoU179], Hopkin and Moss 
[HoMo76], Martin [Ma03], Sipser [S106], and Wood [Wo87]. Bar- 
wise and Etchemendy [BaEt93] is an excellent introduction to Tur- 
ing machines. Interesting articles about the history and application 
of Turing machines and related machines can be found in Herken 
[He88]. Busy beaver machines were first introduced by Rado in 
[Ra62], and information about them can be found in Dewdney 
[De84] and [De93], the article by Brady in Herken [He88], and in 
Wood [Wo87]. 


APPENDIXES 


A discussion of axioms for the real number and for the integers can 
be found in Morash [Mo91]. Detailed treatments of exponential 
and logarithmic functions can be found in calculus books such as 
Apostol [Ap67], Spivak [Sp94], and Thomas and Finney [ThFi96]. 
Pohl and Shaw [PoSh81] use a form of pseudocode that has the 
same features as those described in Appendix 3. Most textbooks 
on algorithms, such as Cormen, Leierson, Rivest, and Stein [Co- 
LeRiSt09] and Kleinberg and Tardos [KITa05], use versions of 
pseudocode similar to the pseudocode in this text. 
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Answers to Odd-Numbered Exercises 


CHAPTER 1 


Section 1.1 


1. a) Yes, T b) Yes, F c) Yes, T d) Yes, F e)No f)No 
3. a) Mei does not have an MP3 player. b) There is pollution 
in New Jersey. c) 2+ 1 43. d) The summer in Maine is not 
hot or itis not sunny. 5. a) Steve does not have more than 
100 GB free disk space on his laptop b) Zach does not block 
e-mails from Jennifer, or he does not block texts from Jennifer 
c) 7-11-13 4999 d) Diane did not ride her bike 100 miles 
onSunday 7. a)Fb)Tc)Td)Te)T 9. a) Sharks have 
not been spotted near the shore. b) Swimming at the New 
Jersey shore is allowed, and sharks have been spotted near the 
shore. ¢c) Swimming at the New Jersey shore is not allowed, 
or sharks have been spotted near the shore. d) If swimming 
at the New Jersey shore is allowed, then sharks have not been 
spotted near the shore. e) If sharks have not been spotted near 
the shore, then swimming at the New Jersey shore is allowed. 
f) If swimming at the New Jersey shore is not allowed, then 
sharks have not been spotted near the shore. g) Swimming 
at the New Jersey shore is allowed if and only if sharks have 
not been spotted near the shore. h) Swimming at the New 
Jersey shore is not allowed, and either swimming at the New 
Jersey shore is allowed or sharks have not been spotted near 
the shore. (Note that we were able to incorporate the paren- 
theses by using the word “either” in the second half of the 
sentence.) ll.a)p Ag b)pA-q c)-=pA-7qd) pvq 
ep>q P~vgrAPp-qd 94< Pp 13.a)7p 
b)pA-¢ )p>qD7p>-q ep>_q fhqrn-p 
gq—>p 18.arAsp b)-=pAgAr or-> (qonrp) 
d) -qA7pAr e)(q>(CrAnp))A7(er Aap) > q) 
f)(p Ar) — -7q_ 17. a) False b) True c) True d) True 
19. a) Exclusive or: You get only one beverage. b) Inclusive 
or: Long passwords can have any combination of symbols. 
c) Inclusive or: A student with both courses is even more qual- 
ified. d) Either interpretation possible; a traveler might wish 
to pay with a mixture of the two currencies, or the store may 
not allow that. 21. a) Inclusive or: It is allowable to take 
discrete mathematics if you have had calculus or computer 
science, or both. Exclusive or: It is allowable to take discrete 
mathematics if you have had calculus or computer science, 
but not if you have had both. Most likely the inclusive or is 
intended. b) Inclusive or: You can take the rebate, or you can 
get a low-interest loan, or you can get both the rebate and a 
low-interest loan. Exclusive or: You can take the rebate, or 
you can get a low-interest loan, but you cannot get both the 
rebate and a low-interest loan. Most likely the exclusive or is 
intended. c) Inclusive or: You can order two items from col- 
umn A and none from column B, or three items from column 
B and none from column A, or five items including two from 
column A and three from column B. Exclusive or: You can 


order two items from column A or three items from column 
B, but not both. Almost certainly the exclusive or is intended. 
d) Inclusive or: More than 2 feet of snow or windchill below 
—100, or both, will close school. Exclusive or: More than 2 
feet of snow or windchill below — 100, but not both, will close 
school. Certainly the inclusive or is intended. 23. a) If the 
wind blows from the northeast, then it snows. Db) If it stays 
warm for a week, then the apple trees will bloom. c) If the Pis- 
tons win the championship, then they beat the Lakers. d) If 
you get to the top of Long’s Peak, then you must have walked 
8 miles. e) If you are world-famous, then you will get tenure 
as a professor. f) If you drive more than 400 miles, then you 
will need to buy gasoline. g) If your guarantee is good, then 
you must have bought your CD player less than 90 days ago. 
h) If the water is not too cold, then Jan will go swimming. 
25. a) You buy an ice cream cone if and only if it is hot out- 
side. b) You win the contest if and only if you hold the only 
winning ticket. c) You get promoted if and only if you have 
connections. d) Your mind will decay if and only if you watch 
television. e) The train runs late if and only if it is a day I 
take the train. 27. a) Converse: “I will ski tomorrow only 
if it snows today.” Contrapositive: “If I do not ski tomorrow, 
then it will not have snowed today.” Inverse: “If it does not 
snow today, then I will not ski tomorrow.” b) Converse: “If I 
come to class, then there will be a quiz.” Contrapositive: “If I 
do not come to class, then there will not be a quiz.” Inverse: 
“Tf there is not going to be a quiz, then I don’t come to class.” 
c) Converse: “A positive integer is a prime if it has no divisors 
other than 1 and itself.’ Contrapositive: “If a positive integer 
has a divisor other than | and itself, then it is not prime.” In- 
verse: “If a positive integer is not prime, then it has a divisor 
other than | and itself’? 29. a)2 b)16 c)64 d) 16 
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41. The first clause is true if and only if at least one of p, g, and 
r is true. The second clause is true if and only if at least one of 
the three variables is false. Therefore the entire statement is 
true if and only if there is at least one T and one F among the 
truth values of the variables, in other words, that they don’t all 
have the same truth value. 43. a) Bitwise OR is 111 1111; 
bitwise AND is 000 0000; bitwise XOR is 111 1111. b) Bitwise 
OR is 1111 1010; bitwise AND is 1010 0000; bitwise XOR is 
0101 1010. c) Bitwise OR is 10 0111 1001; bitwise AND is 
00 0100 0000; bitwise XOR is 10 0011 1001. d) Bitwise OR 
is 11 1111 1111; bitwise AND is 00 0000 0000; bitwise XOR 
is 11 1111 1111. 45.0.2,0.6 47. 0.8,0.6 49. a) The 
99th statement is true and the rest are false. b) Statements 
1 through 50 are all true and statements 51 through 100 are 
all false. c) This cannot happen; it is a paradox, showing that 
these cannot be statements. 


Section 1.2 


leroa 3.g7>(rA(Am)A (Ab) 5.e> (AA (bV 
P)Ar) 7T.aq>p b)qgA7p Og> p D-~q>-7p 
9. Not consistent 11. Consistent 13. NEW AND JER- 
SEY AND BEACHES, (JERSEY AND BEACHES) NOT 
NEW 15. “If I were to ask you whether the right branch 
leads to the ruins, would you answer yes?” 17 If the first 
professor did not want coffee, then he would know that the an- 
swer to the hostess’s question was “no.” Therefore the hostess 
and the remaining professors know that the first professor did 
want coffee. Similarly, the second professor must want coffee. 
When the third professor said “no,” the hostess knows that the 
third professor does not want coffee. 19. A is a knight and 
Bisaknave. 21. Aisaknightand Bisaknight. 23. Ais 
aknave and Bisaknight. 25. A isthe knight, B is the spy, C 
istheknave. 27. Aisthe knight, B is the spy, C is the knave. 
29. Any of the three can be the knight, any can be the spy, any 
can be the knave. 31. No solutions 33. In order of de- 
creasing salary: Fred, Maggie, Janice 35. The detective can 
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determine that the butler and cook are lying but cannot deter- 
mine whether the gardener is telling the truth or whether the 
handyman is telling the truth. 37. The Japanese man owns 
the zebra, and the Norwegian drinks water. 39. One honest, 
49 corrupt 41. a) -(pA(qV —7r))b) (=p) A(-q))V (PAT) 
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Section 1.3 


1. The equivalences follow by showing that the appropriate 
pairs of columns of this table agree. 


cia Ae T F r T T 
F F F i F F 
a) 9g Dp a|pAq|aap 
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T| F F 
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7. a) Jan is not rich, or Jan is not happy. b) Carlos will not 
bicycle tomorrow, and Carlos will not run tomorrow. c) Mei 
does not walk to class, and Mei does not take the bus to class. 
d) Ibrahim is not smart, or Ibrahim is not hard working. 


mea) (pAq)—> Pp 


Mods 
mamas 
BHAA 


S-4 Answers to Odd-Numbered Exercises 


=p>(p>4q) 


(pAq) > (p> q) 


“(p> q)—> Pp 


“(p> q)> 7-9 


11. In each case we will show that if the hypothesis is true, 
then the conclusion is also. a) If the hypothesis p A q is true, 
then by the definition of conjunction, the conclusion p must 
also be true. b) If the hypothesis p is true, by the definition 
of disjunction, the conclusion p V q is also true. c) If the 
hypothesis —p is true, that is, if p is false, then the conclusion 
Pp — qistrue. d) If the hypothesis p A q is true, then both 
p and gq are true, so the conclusion p — gq is also true. e) If 
the hypothesis =(p — q) is true, then p — gq is false, so 
the conclusion p is true (and q is false). f) If the hypothesis 
=(p — q) is true, then p — gq is false, so p:p is true and q is 
false. Hence, the conclusion —g is true. 13. That the fourth 
column of the truth table shown is identical to the first column 
proves part (a), and that the sixth column is identical to the 
first column proves part (b). 


pV (PAq) PA (PV 4) 
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15. Itisatautology. 17. Each of these is true precisely when 
p and q have opposite truth values. 19. The proposition 


=p < q is true when —p and q have the same truth val- 
ues, which means that p and q have different truth values. 
Similarly, p <> —q is true in exactly the same cases. There- 
fore, these two expressions are logically equivalent. 21. The 
proposition =(p <> q) is true when p < q is false, which 
means that p and q have different truth values. Because this is 
precisely when —p <> q is true, the two expressions are logi- 
cally equivalent. 23. For(p — r)A(q — r) tobe false, one 
of the two conditional statements must be false, which hap- 
pens exactly when r is false and at least one of p and gq is true. 
But these are precisely the cases in which p V q is true andr is 
false, which is precisely when (p V q) — r is false. Because 
the two propositions are false in exactly the same situations, 
they are logically equivalent. 25. For (p > r) Vv (¢q > r) 
to be false, both of the two conditional statements must be 
false, which happens exactly when r is false and both p and 
q are true. But this is precisely the case in which p A gq is 
true and r is false, which is precisely when (p A gq) > r 
is false. Because the two propositions are false in exactly the 
same situations, they are logically equivalent. 27. This fact 
was observed in Section 1 when the biconditional was first 
defined. Each of these is true precisely when p and q have the 
same truth values. 29. The last column is all Ts. 
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31. These are not logically equivalent because when p, q, and 
r are all false, (p — gq) — r is false, but p > (q — r) is 
true. 33. Many answers are possible. If we let r be true and 
Pp. q, and s be false, then (p > q) > (r > s) will be false, 
but (p > r) > (¢ > s) willbetrue. 35.a) pV 7~=q V -r 
b) (pV qvVvr)As ©) (pAT)V(GAF) — 37. If we take duals 
twice, every V changes to an A and then back to an V, every 
/ changes to an V and then back to an A, every T changes to 
an F and then back to a T, every F changes to a T and then 
back to an F. Hence, (s*)* = s. 39. Let p and q be equiv- 
alent compound propositions involving only the operators A, 
V, and —, and T and F. Note that =p and -q are also equiv- 
alent. Use De Morgan’s laws as many times as necessary to 
push negations in as far as possible within these compound 
propositions, changing Vs to As, and vice versa, and chang- 
ing Ts to Fs, and vice versa. This shows that =p and 7g are 
the same as p* and g* except that each atomic proposition 
p; within them is replaced by its negation. From this we can 
conclude that p* and g* are equivalent because =p and —g 


are. 41.(pAqGAr-7r)V (pA 7qATr)V (ApAG Ar) 
43. Given a compound proposition p, form its truth table 
and then write down a proposition q in disjunctive nor- 
mal form that is logically equivalent to p. Because qg in- 
volves only —, A, and V, this shows that these three op- 
erators form a functionally complete set. 45. By Exercise 
43, given a compound proposition p, we can write down a 
proposition q that is logically equivalent to p and involves 
only —, A, and Vv. By De Morgan’s law we can eliminate all 
the Ws by replacing each occurrence of pi A p2 A--+ A pn 
with =(>=py Vap2 V---V apy). 47.7(p A gq) is true 
when either p or q, or both, are false, and is false 
when both p and q are true. Because this was the defi- 
nition of p |g, the two compound propositions are logi- 
cally equivalent. 49.—(p Vv q) is true when both p and 
q are false, and is false otherwise. Because this was 
the definition of p | q, the two are logically equivalent. 
Sl. (pl pla) (pl p)t@) 53.This follows im- 
mediately from the truth table or definition of p | q. 
55.16 57. If the database is open, then either the sys- 
tem is in its initial state or the monitor is put in a closed 
state. 59.All nine 61. a) Satisfiable b) Not satisfiable 
c) Not satisfiable 63. Use the same propositions as were 
given in the text for a 9 x 9 Sudoku puzzle, with the vari- 
ables indexed from | to 4, instead of from 1 to 9, and 
with a similar change for the propositions for the 2 x 2 
blocks: \ 9 Aseo nat Vier Vjn1 PQr + i, 28 + jn) 
65. ve , PG, j, ) asserts that column j contains the number 
n, SO es Vey p(i, j, n) asserts that column j contains all 
9 numbers; therefore Nit ome Ve p(i, j, n) asserts that 
every column contains every number. 
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l.a)T b)T c)F 3.a)T b)F c)F d)F  5.a) There 
is a student who spends more than 5 hours every weekday 
in class. b) Every student spends more than 5 hours ev- 
ery weekday in class. c) There is a student who does not 
spend more than 5 hours every weekday in class. d) No 
student spends more than 5 hours every weekday in class. 
7. a) Every comedian is funny. b) Every person is a funny 
comedian. c) There exists a person such that if she or he is 
a comedian, then she or he is funny. d) Some comedians 
are funny. 9.a)dx(P(x) A Q(x)) b) Aax(P(x) A 7=Q(4)) 
c) Vx(P(x) Vv Q(x)) d) Vx-(P(x) Vv Q(x)) Ll.a)T b)T 
oF dFe)T fF 13.aT DDT oT d@T 15.a)T 
b)Fe)T dF 17.a) P(O) v PC) v P(2)v P(3) v P(A) 
b) P(O) A PL) A P(2)A P(3) A P(A) ©) =P(0O) V=P(1) Vv 
=P(2) Vv =P(3) Vv =P(4) d)-P(O) A =P(1) A 
=P(2) A=P(3) A =P(4) e) -(P(O) V PCI) Vv P(2) Vv 
P(3) Vv P(4)) f) =(P(0) A PQ) A PQ) A PB) A P(A) 
19.a) PA) Vv P(2)V P(3)V P(4)V P(5) ob) PO)AP(Q)A 
P(3)A P(4)A P(5) ¢) -(P (1) VP(2) VP(3) VP(4) VP(5)) 
d) -(P(1) A P(2) A P(3) A P(4) A P(S)) e) (PU) A 
P(2) A P(4) A P(5)) V (APC) V =P) Vv AP(3) Vv 
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=P (4) v =P(5)) 21. Many answers are possible. a) All 
students in your discrete mathematics class; all students in 
the world b) All United States senators; all college football 
players c) George W. Bush and Jeb Bush; all politicians 
in the United States d) Bill Clinton and George W. Bush; 
all politicians in the United States 23. Let C(x) be the 
propositional function “x is in your class.” a) 4x H(x) and 
dx(C(x) A H(x)), where H(x) is “x can speak Hindi” 
b) Vx F(x) and Vx(C(x) > F(x)), where F(x) is “x is 
friendly” ce) dx-B(x) and 4x(C(x) A =B(x)), where B(x) is 
“x was born in California” d) 4x M(x) and 4x (C(x) A M(x)), 
where M(x) is “x has been in a movie” e)Vx—-L(x) and 
Vx(C(x) — -=L(x)), where L(x) is “x has taken a course 
in logic programming” 25. Let P(x) be “x is perfect”; 
let F(x) be “x is your friend”; and let the domain be all 
people. a) Vx —P(x) b)-Vx P(x) ©) Vx(F(x) > P(x)) 
d) Ax(F(x) A P(x)) e) Vx(F (x) A P(x)) or (Vx F(x)) A 
(Wx P(x)) f) (AVx F(x)) Vv (Ax =P(x)) 27. Let Y(x) be 
the propositional function that x is in your school or class, as 
appropriate. a) If we let V(x) be “x has lived in Vietnam,” 
then we have 4x V (x) if the domain is just your schoolmates, 
or dx(Y(x) A V(x)) if the domain is all people. If we let 
D(x, y) mean that person x has lived in country y, then we 
can rewrite this last one as 4x(Y(x) A D(x, Vietnam)). b) If 
we let H(x) be “x can speak Hindi,” then we have 4x—A (x) 
if the domain is just your schoolmates, or 4x (Y (x) A -H(x)) 
if the domain is all people. If we let S(x, y) mean that per- 
son x can speak language y, then we can rewrite this last 
one as 4x(Y(x) A —S(x, Hindi)). c) If we let J(x), P(x), 
and C(x) be the propositional functions asserting x’s knowl- 
edge of Java, Prolog, and C++, respectively, then we have 
dx(J(x) A P(x) A C(x)) if the domain is just your school- 
mates, or dx(Y(x) A J(x) A P(x) A C(x)) if the domain 
is all people. If we let K(x, y) mean that person x knows 
programming language y, then we can rewrite this last one as 
Ax (¥ (x) A K(x, Java) A K (x, Prolog) AK (x, C++)). d) If 
we let T(x) be “x enjoys Thai food,” then we have Vx T (x) 
if the domain is just your classmates, or Vx(Y(x) > T(x)) 
if the domain is all people. If we let E(x, y) mean that 
person x enjoys food of type y, then we can rewrite this 
last one as Vx(Y(x) — E(x, Thai)). e) If we let H(x) 
be “x plays hockey,” then we have 4x —H(x) if the do- 
main is just your classmates, or 4x(Y(x) A =A (x)) if 
the domain is all people. If we let P(x, y) mean that per- 
son x plays game y, then we can rewrite this last one as 
dx (¥(x) A =P (x, hockey)). 29. Let T(x) mean that x is a 
tautology and C(x) mean that x is acontradiction. a) 4x T (x) 
b) Vx(C (x) > T(>x)) e) axdy(AT (x) ARC(X) ARTO) A 
SC(y)AT(x V yy) d) ¥xVy(T(x) AT(y)) > Tey) 
31. a) Q(0,0,0) A Q(0,1,0) b) Q(0,1,1) V@d, 1, 1) v 
Q(2, 1,1) e) =~Q(0,0,0) Vv =Q@(0, 0,1) d) =Q(0, 0, 1) v 
=0(1,0,1)V-Q(2,0,1) 33. a) Let T(x) be the predicate 
that x can learn new tricks, and let the domain be old dogs. 
Original is dx T (x). Negation is Vx —T (x): “No old dogs can 
learn new tricks.” b) Let C(x) be the predicate that x knows 
calculus, and let the domain be rabbits. Original is ~=4x C(x). 
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Negation is 4x C(x): “There is a rabbit that knows calculus.” 
c) Let F(x) be the predicate that x can fly, and let the domain 
be birds. Original is Vx F(x). Negation is 4x —F (x): “There is 
a bird who cannot fly.” d) Let T (x) be the predicate that x can 
talk, and let the domain be dogs. Original is ~4x T(x). Nega- 
tion is ax T(x): “There is a dog that talks.” e) Let F(x) and 
R(x) be the predicates that x knows French and knows Rus- 
sian, respectively, and let the domain be people in this class. 
Original is —=3x (F(x) A R(x)). Negation is dx (F(x) A R(x)): 
“There is someone in this class who knows French and Rus- 
sian.’ 35. a) There isnocounterexample. b) x =0 c) x =2 
37. a) Vx ((F (x, 25,000) V S(x, 25)) — E(x)), where E(x) 
is “Person x qualifies as an elite flyer in a given year,” F(x, y) 
is “Person x flies more than y miles in a given year,’ and 
S(x, y) is “Person x takes more than y flights in a given year” 
b) Vx((M(x) AT (x, 3) V (FM (x) AT (x, 3.5))) > Q(X), 
where Q(x) is “Person x qualifies for the marathon,” 
M(x) is “Person x is a man,” and T(x, y) is “Person x 
has run the marathon in less than y hours” ¢e)M — 
((H(60) Vv (H(45) A T)) A Vy G(B, y)), where M is the 
proposition “The student received a masters degree,” H (x) is 
“The student took at least x course hours,” T is the proposition 
“The student wrote a thesis,” and G(x, y) is “The person got 
grade x or higher in course y” d) 4x ((T (x, 21) A G(x, 4.0)), 
where T(x, y) is “Person x took more than y credit hours” 
and G(x, p) is “Person x earned grade point average p” 
(we assume that we are talking about one given semester) 
39. a) If there is a printer that is both out of service and 
busy, then some job has been lost. b) If every printer is 
busy, then there is a job in the queue. c) If there is a job 
that is both queued and lost, then some printer is out of ser- 
vice. d) If every printer is busy and every job is queued, 
then some job is lost. 41. a) (Ax F(x, 10)) — Ax S(x), 
where F(x, y) is “Disk x has more than y kilobytes of 
free space,’ and S(x) is “Mail message x can be saved” 
b) (Ax A(x)) > Vx(Q(x) > T(x)), where A(x) is “Alert x 
is active,’ Q(x) is “Message x is queued,” and T(x) is “Mes- 
sage x is transmitted” c)Vx((x # main console) > T(x)), 
where T(x) is “The diagnostic monitor tracks the status of 
system x” d) Vx(-—L(x) — B(x)), where L(x) is “The host 
of the conference call put participant x on a special list” and 
B(x) is “Participant x was billed” 43. They are not equiva- 
lent. Let P (x) be any propositional function that is sometimes 
true and sometimes false, and let Q(x) be any propositional 
function that is always false. Then Vx(P(x) > Q(x)) is false 
but Vx P(x) — VWxQ(x) is true. 45. Both statements are 
true precisely when at least one of P(x) and Q(x) is true for 
at least one value of x in the domain. 47. a) If A is true, 
then both sides are logically equivalent to Vx P(x). If A is 
false, the left-hand side is clearly false. Furthermore, for every 
x, P(x) A A is false, so the right-hand side is false. Hence, 
the two sides are logically equivalent. b) If A is true, then 
both sides are logically equivalent to dx P(x). If A is false, 
the left-hand side is clearly false. Furthermore, for every x, 
P(x) A Ais false, so dx(P(x) A A) is false. Hence, the two 
sides are logically equivalent. 49. We can establish these 
equivalences by arguing that one side is true if and only if the 


other side is true. a) Suppose that A is true. Then for each x, 
P(x) — A is true; therefore the left-hand side is always true 
in this case. By similar reasoning the right-hand side is always 
true in this case. Therefore, the two propositions are logically 
equivalent when A is true. On the other hand, suppose that A 
is false. There are two subcases. If P(x) is false for every x, 
then P(x) — AA is vacuously true, so the left-hand side is 
vacuously true. The same reasoning shows that the right-hand 
side is also true, because in this subcase 4x P(x) is false. For 
the second subcase, suppose that P(x) is true for some x. Then 
for that x, P(x) — Ais false, so the left-hand side is false. The 
right-hand side is also false, because in this subcase 4x P (x) 
is true but A is false. Thus in all cases, the two propositions 
have the same truth value. b) If A is true, then both sides 
are trivially true, because the conditional statements have 
true conclusions. If A is false, then there are two subcases. If 
P(x) is false for some x, then P(x) — A is vacuously true 
for that x, so the left-hand side is true. The same reasoning 
shows that the right-hand side is true, because in this subcase 
Vx P(x) is false. For the second subcase, suppose that P (x) is 
true for every x. Then for every x, P(x) — A is false, so the 
left-hand side is false (there is no x making the conditional 
statement true). The right-hand side is also false, because it is 
a conditional statement with a true hypothesis and a false con- 
clusion. Thus in all cases, the two propositions have the same 
truth value. 51. To show these are not logically equivalent, 
let P(x) be the statement “x is positive,’ and let Q(x) be 
the statement “x is negative” with domain the set of integers. 
Then 3x P(x) A Ax Q(x) is true, but dx(P(x) A Q(x)) is 
false. 53.a) True b) False, unless the domain consists of 
just one element c) True 55.a) Yes b) No c) juana, kiko 
d) math273, cs301 =e) juana, kiko 57. sibling (X,Y) 
:- mother(M,X), mother(M,Y), father(F,X), 
father(F,Y) 59.a)Vx(P(x) > =3Q(x)) 
b) Vx(Q(x) — R(x)) ec) Vx(P(x) — -=R(x)) d) The 
conclusion does not follow. There may be vain profes- 
sors, because the premises do not rule out the possibil- 
ity that there are other vain people besides ignorant ones. 
6l.a)Vx(P(x) — =Q(x)) b)Vx(R(x) —> -=S(x)) 
ce) Vx(A= Q(x) > S(x)) d)Vx(P (x) — -=R(x)) e) The 
conclusion follows. Suppose x is a baby. Then by the first 
premise, x is illogical, so by the third premise, x is despised. 
The second premise says that if x could manage a crocodile, 
then x would not be despised. Therefore, x cannot manage a 
crocodile. 


Section 1.5 


1. a) For every real number x there exists a real number y 
such that x is less than y. b) For every real number x and real 
number y, if x and y are both nonnegative, then their product 
is nonnegative. c) For every real number x and real number 
y, there exists areal number z such thatxy =z. 3. a) There 


is some student in your class who has sent a message to 
some student in your class. b) There is some student in your 
class who has sent a message to every student in your class. 
c) Every student in your class has sent a message to at least 
one student in your class. d) There is a student in your class 
who has been sent a message by every student in your class. 
e) Every student in your class has been sent a message from 
at least one student in your class. f) Every student in the class 
has sent a message to every student in the class. 5. a) Sarah 
Smith has visited www.att.com. b) At least one person has 
visited www.imdb.org. c) Jose Orez has visited at least one 
website. d) There is a website that both Ashok Puri and 
Cindy Yoon have visited. e) There is a person besides David 
Belcher who has visited all the websites that David Belcher 
has visited. f) There are two different people who have visited 
exactly the same websites. 7. a) Abdallah Hussein does not 
like Japanese cuisine. b) Some student at your school likes 
Korean cuisine, and everyone at your school likes Mexican 
cuisine. c) There is some cuisine that either Monique Ar- 
senault or Jay Johnson likes. d) For every pair of distinct 
students at your school, there is some cuisine that at least one 
them does not like. e) There are two students at your school 
who like exactly the same set of cuisines. f) For every pair 
of students at your school, there is some cuisine about which 
they have the same opinion (either they both like it or they 
both do not like it), 9.a)VxL(x, Jerry) b) VxdyL(x, y) 
ce) dyVxL(x, y) d)Vxdy-L(x, y) e)dx-L (Lydia, x) 
f) axVy-L(y, x) g)dx(VyLGo, x) A Vz(VwLl(w, z)) => 
z= x)) h)aAxvdy@ # yA L(Lynn, x) A L(Lynn, y) A 
Vz(L(Lynn, z) > (zg =xVz=y))) DVxL(x,x) JaxVy 
(L(x,y) <= x = y) 11.a) A(Lois, Professor Michaels) 
b) Vx(S(x) — A(x, Professor Gross)) ¢) Vx(F (x) > (A(x, 
Professor Miller) V A(Professor Miller, x))) d) dx(S(x) A 
Vy(F(y) = A(x, y)) e)AX(FQX) A Vy(S(Qy) > 
AA(y,x))) £) Vy(F(y) > 3x(S(x) V A(a,y))) 8) Sx(F (x) A 
Vy(F(y) A Gy # x)) >Ay)))  W)AX(SQX) A 
Vy(F(y) —> -7A(y, x))) 13.a)-M (Chou, Koko) 
b) —M (Arlene, Sarah) \ =7 (Arlene, Sarah) ¢) —M (Debo- 
rah, Jose) d) Vx M(x, Ken) e) Vx-7 (x, Nina) f) Vx (T-x, Avi) 
VM(x, Avi) g) axVy(y Ax > M(x, y)) h) axVy(y 4 
x > (M(x, y) Vv T(x, y))) Avaya A yA M(x, y) A 
M(y,x)) j)axM(x,x) k) AxVy(@e A y > (“M(x, y) A 
“T(y, x))) DVxGya # y A (My, x) Vv Ty, x))) 
m) Axdy(x # yA M(x, y) AT(y,x)) mAxdy@ A yA 
Wiz Fx Az#y) > (M(x,2) V MVD VT DV 
T(y,z)))) 15. a) Vx P(x), where P(x) is “x needs a course 
in discrete mathematics” and the domain consists of all com- 
puter science students b) 4x P(x), where P(x) is “x owns 
a personal computer” and the domain consists of all students 
in this class ¢) VxdyP(x, y), where P(x, y) is “x has taken 
y,” the domain for x consists of all students in this class, 
and the domain for y consists of all computer science classes 
d) SxSy P(x, y), where P(x, y) and domains are the same as 
in part (c) e) VxVyP(x, y), where P(x, y) is “x has been 
in y,” the domain for x consists of all students in this class, 
and the domain for y consists of all buildings on campus 
f) SxdyVz(P(z, y) — Q(x, z)), where P(z, y) is “z is in 
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y” and Q(x, z) is “x has been in z”; the domain for x con- 
sists of all students in the class, the domain for y consists 
of all buildings on campus, and the domain of z consists of 
all rooms. g) VxVydz(P(z, y) A Q(x, z)), with same en- 
vironment as in part (f) 17. a) Vudm(A(u, m) A Va(n # 
m —> —A(u,n))), where A(u, m) means that user u has 
access to mailbox m _b) dpVe(H(e) A S(p, running)) 
— S (kernel, working correctly), where H(e) means that 
error condition e is in effect and S(x, y) means that the 
status of x is y c)VuVs(E(s, .edu) — A(u, s)), where 
E(s, x) means that website s has extension x, and A(u, s) 
means that user wu can access website s d)dxdy(x F 
y A V2(Ws Miz,s)) eo ( = x Vz = y))), 
where M (a, b) means that system a monitors remote server b 
19. a) VxVy((x < 0) A (y < 0) > (4+ y < 0)) Db) AVxVy 
(x >0) A (y>0) > (@—y>0)) ©) ¥xVy (x? + y? = 
(x + y)*) d) VxVy (Jxy| = |xlly]) 21. Wxdadbacdd (x > 
0) > x = a2 4+ b? +c? + d), where the domain consists 
ofallintegers 23.a) Vx Vy ((x < 0) A (y < 0) > (vy > 
0)) b) V¥x(x —x = 0) ©) Vxdadb(a 4b AVC(CA= XO 
(c =aVc=b))) dVx((x < 0) > 7ASy = y?)) 
25. a) There is a multiplicative identity for the real numbers. 
b) The product of two negative real numbers is always a pos- 
itive real number. c) There exist real numbers x and y such 
that x? exceeds y but x is less than y. d) The real numbers are 
closed under the operation of addition. 27. a) True b) True 
c) True d) True e) True f) False g) False h) True i) False 
29.a) P(,1l) A Pd,2) A Pd,3) A PQ,1) A P(2,2) A 
P(2,3) A PB, 1) A PB, 2) A PB, 3) b) Pd, 1) Vv 
Pd,2)Vv Pd,3) Vv PQ, 1) Vv PQ, 2) Vv P(,3) Vv PB3,1I) Vv 
P@G, 2) Vv P(3, 3) ©) (Pd, 1) A Pd, 2) A Pd, 3)) Vv 
(P(2,1)A PQ, 2) A P(2, 3))V(P(3, 1) A PG, 2) A P(3, 3)) 
d) (Pd, 1) v PQ,1) Vv PG, 1) A (Pd, 2) Vv PQ, 2) Vv 
P(3,2)) A (PC, 3) Vv P(2, 3) V PG, 3)) 31. a) dxVydz aT 
(x, y,z) b)AxVy-P(x, y) A AxVy = Q(x, y) ce) AxVy 
(4P(x, y) VWz7R(x, y, z)) d) AxVy(P(x, y) A7Q(x, y)) 
33. a) dxdy-P(x, y) b)AyVx-P(x, y) c) dydx(-P(x, 
y) A 7Q(x, y)) d) (VxVyP(x, y)) V @xdy7Q(, y)) 
e) dx (Vydz-P (x,y,z) V Vzdy>P(x, y,z)) 35. Any do- 
main with four or more members makes the statement true; 
any domain with three or fewer members makes the state- 
ment false. 37.a) There is someone in this class such 
that for every two different math courses, these are not 
the two and only two math courses this person has taken. 
b) Every person has either visited Libya or has not visited 
a country other than Libya. c) Someone has climbed ev- 
ery mountain in the Himalayas. d) There is someone who 
has neither been in a movie with Kevin Bacon nor has been 
in a movie with someone who has been in a movie with 
Kevin Bacon. 39.a)x = 2,y = —2 b)x = -4 
ox =17,y =—-1 41.VxVyVz((x-y)-z = x-(y-Zz)) 
43.VmVb(m #4 0 > Ax(mx + b= 0 A Vw(mw + b = 
0 - w = x))) 45.a)True_ b) False’ c) True 
47. a=(AxVy P(x, y)) @ Vx(AVyP(x, y)) << WxdyaP (x,y) 
49. a) Suppose that Vx P(x) A 4x Q(x) is true. Then P(x) is 
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true for all x and there is an element y for which Q(y) is true. 
Because P(x) A Q(y) is true for all x and there is a y for which 
Q(y) is true, Vxdy(P(x) A Q(y)) is true. Conversely, sup- 
pose that the second proposition is true. Let x be an element 
in the domain. There is a y such that Q(y) is true, so dx Q(x) 
is true. Because Vx P(x) is also true, it follows that the first 
proposition is true. b) Suppose that Vx P(x) V 4x Q(x) is 
true. Then either P(x) is true for all x, or there exists a y for 
which Q(y) is true. In the former case, P(x) V Q(y) is true 
for all x, so Vxdy(P(x) V Q(y)) is true. In the latter case, 
Q(y) is true for a particular y, so P(x) V Q(y) is true for all 
x and consequently Vxdy(P(x) V Q(y)) is true. Conversely, 
suppose that the second proposition is true. If P (x) is true for 
all x, then the first proposition is true. If not, P(x) is false for 
some x, and for this x there must be a y such that P(x) V Q(y) 
is true. Hence, Q(y) must be true, so dy Q(y) is true. It fol- 
lows that the first proposition must hold. 51. We will show 
how an expression can be put into prenex normal form (PNF) 
if subexpressions in it can be put into PNF. Then, working 
from the inside out, any expression can be put in PNF. (To 
formalize the argument, it is necessary to use the method of 
structural induction that will be discussed in Section 5.3.) By 
Exercise 45 of Section 1.4, we can assume that the proposition 
uses only V and — as logical connectives. Now note that any 
proposition with no quantifiers is already in PNF. (This is the 
basis case of the argument.) Now suppose that the proposition 
is of the form Qx P(x), where Q is a quantifier. Because P (x) 
is a shorter expression than the original proposition, we can 
put it into PNF. Then Qx followed by this PNF is again in PNF 
and is equivalent to the original proposition. Next, suppose 
that the proposition is of the form —P. If P is already in PNF, 
we slide the negation sign past all the quantifiers using the 
equivalences in Table 2 in Section 1.4. Finally, assume that 
proposition is of the form P V Q, where each of P and Q isin 
PNEF. If only one of P and Q has quantifiers, then we can use 
Exercise 46 in Section 1.4 to bring the quantifier in front of 
both. If both P and Q have quantifiers, we can use Exercise 
45 in Section 1.4, Exercise 48, or part (b) of Exercise 49 to 
rewrite P V Q with two quantifiers preceding the disjunction 
of a proposition of the form R Vv S, and then put R Vv S into 
PNF. 


Section 1.6 


1. Modus ponens; valid; the conclusion is true, because 
the hypotheses are true. 3. a) Addition b) Simplification 
c) Modus ponens d) Modus tollens e) Hypothetical syllo- 
gism 5. Let w be “Randy works hard,” let d be “Randy is a 
dull boy,” and let 7 be “Randy will get the job.” The hypothe- 
ses are w, w > d,andd — —j]. Using modus ponens and the 
first two hypotheses, d follows. Using modus ponens and the 
last hypothesis, = 7, which is the desired conclusion, “Randy 


will not get the job,” follows. 7. Universal instantiation is 
used to conclude that “If Socrates is a man, then Socrates is 
mortal.” Modus ponens is then used to conclude that Socrates 
is mortal. 9. a) Valid conclusions are “I did not take Tues- 
day off,” “I took Thursday off,’ “It rained on Thursday.” b) “I 
did not eat spicy foods and it did not thunder” is a valid con- 
clusion. c) “I am clever” is a valid conclusion. d) “Ralph 
is not a CS major” is a valid conclusion. e) “That you buy 
lots of stuff is good for the U.S. and is good for you” is a 
valid conclusion. f) “Mice gnaw their food” and “Rabbits 
are not rodents” are valid conclusions. 11. Suppose that 
P1, P2;+--++ Pn are true. We want to establish that g — r 
is true. If g is false, then we are done, vacuously. Otherwise, 
q is true, so by the validity of the given argument form (that 
whenever p}, P2,---, Pn, g are true, then r must be true), we 
know that r is true. 13. a) Let c(x) be “‘x is in this class,” 
J (x) be “x knows how to write programs in JAVA,” and h(x) 
be “x can get a high-paying job.” The premises are c(Doug), 
j(Doug), Vx(j(x) — h(x)). Using universal instantiation 
and the last premise, j (Doug) — h(Doug) follows. Applying 
modus ponens to this conclusion and the second premise, 
h(Doug) follows. Using conjunction and the first premise, 
c(Doug) A h(Doug) follows. Finally, using existential gener- 
alization, the desired conclusion, 4x(c(x) A h(x)) follows. 
b) Let c(x) be “x is in this class,” w(x) be “x enjoys whale 
watching,” and p(x) be “x cares about ocean pollution.” 
The premises are dx(c(x) A w(x)) and Vx(w(x) > p(x)). 
From the first premise, c(y) A w(y) for a particular per- 
son y. Using simplification, w(y) follows. Using the sec- 
ond premise and universal instantiation, w(y) — p(y) 
follows. Using modus ponens, p(y) follows, and by con- 
junction, c(y) A p(y) follows. Finally, by existential gen- 
eralization, the desired conclusion, dx(c(x) A p(x)), fol- 
lows. c) Let c(x) be “x is in this class,’ p(x) be “x owns 
a PC,” and w(x) be “x can use a word-processing pro- 
gram.” The premises are c(Zeke), Vx(c(x) — p(x)), and 
Vx (p(x) — w(x)). Using the second premise and universal 
instantiation, c(Zeke) — p(Zeke) follows. Using the first 
premise and modus ponens, p(Zeke) follows. Using the third 
premise and universal instantiation, p(Zeke) > w(Zeke) 
follows. Finally, using modus ponens, w(Zeke), the desired 
conclusion, follows. d) Let j(x) be “x is in New Jersey,” 
Ff (x) be “x lives within 50 miles of the ocean,” and s(x) be 
“x has seen the ocean.” The premises are Vx(j(x) > f(x)) 
and dx(j(x) A —s(x)). The second hypothesis and existen- 
tial instantiation imply that j(y) A —s(y) for a particular 
person y. By simplification, j(y) for this person y. Using 
universal instantiation and the first premise, j(y) > f(y), 
and by modus ponens, f(y) follows. By simplification, —s(y) 
follows from j(y) A—s(y). So f(y) A >s(y) follows by con- 
junction. Finally, the desired conclusion, 4x(f (x) A >s(x)), 
follows by existential generalization. 15. a) Correct, using 
universal instantiation and modus ponens_b) Invalid; fallacy 
of affirming the conclusion c) Invalid; fallacy of denying 
the hypothesis d) Correct, using universal instantiation and 
modus tollens 17. We know that some x exists that makes 


H (x) true, but we cannot conclude that Lola is one such x. 
19. a) Fallacy of affirming the conclusion b) Fallacy of beg- 
ging the question c) Valid argument using modus tollens 
d) Fallacy of denying the hypothesis 21. By the second 
premise, there is some lion that does not drink coffee. Let 
Leo be such a creature. By simplification we know that Leo 
is a lion. By modus ponens we know from the first premise 
that Leo is fierce. Hence, Leo is fierce and does not drink 
coffee. By the definition of the existential quantifier, there 
exist fierce creatures that do not drink coffee, that is, some 
fierce creatures do not drink coffee. 23. The error occurs in 
step (5), because we cannot assume, as is being done here, 
that the c that makes P true is the same as the c that makes Q 
true. 25. We are given the premises Vx(P(x) — Q(x)) and 
—=Q(a). We want to show —P(a). Suppose, to the contrary, 
that —P(a) is not true. Then P(q) is true. Therefore by uni- 
versal modus ponens, we have Q(a). But this contradicts the 
given premise —Q(a). Therefore our supposition must have 
been wrong, and so —P (a) is true, as desired. 


27. Step Reason 
1.Vx(P(x) A R(x)) Premise 
2. P(a) A R(a) Universal instantiation from (1) 


3. P(a) Simplification from (2) 
4. Vx(P(x) > Premise 
(Q(x) A S(x))) 
5. O(a) A S(a) Universal modus ponens from (3) 
and (4) 
6. S(a) Simplification from (5) 
7. R(a) Simplification from (2) 


8. R(a) A S(a) 
9. Vx(R(x) A S(x)) 


Conjunction from (7) and (6) 
Universal generalization from (5) 


29. Step Reason 
1. dx P(x) Premise 
2. 3P(c) Existential instantiation from (1) 
3. Vx(P(x) V Q(x)) Premise 
4. P(c) V Q(c) Universal instantiation from (3) 
5. O(c) Disjunctive syllogism from (4) 
and (2) 


6. Vx(-=Q(x) V S(x)) Premise 
7. 3Q(c) V S(c) Universal instantiation from (6) 
8. S(c) Disjunctive syllogism from (5) 
and (7) 
9. Vx(R(x) > —S(x)) Premise 
10. R(c) > -=S(c) Universal instantiation from (9) 
11. =R(c) Modus tollens from (8) and (10) 
12. dx-R(x) Existential generalization from 
(11) 
31. Let p be “It is raining”; let g be “Yvette has her umbrella’; 
let r be “Yvette gets wet.” Assumptions are =p V qg, -=q V -r, 
and p V —r. Resolution on the first two gives =p V —r. Res- 
olution on this and the third assumption gives =r, as desired. 
33. Assume that this proposition is satisfiable. Using resolu- 
tion on the first two clauses enables us to conclude q V q; in 
other words, we know that q has to be true. Using resolution on 
the last two clauses enables us to conclude —q V —q; in other 
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words, we know that —g has to be true. This is a contradiction. 
So this proposition is not satisfiable. 35. Valid 


Section 1.7 


1. Let n = 2k + 1 andm = 2/ + 1 be odd inte- 
gers. Then n+m=2(k+/+1) is even. 3. Suppose that 
n is even. Then n = 2k for some integer k. Therefore, 
n2 = (2k)* = 4k? = 2(2k?). Because we have written n” 
as 2 times an integer, we conclude that n2iseven. 5. Direct 
proof: Suppose that m+n andn-+ p are even. Thenm-+n = 2s 
for some integer s andn + p = 2t for some integer ¢. If we 
add these, we get m+ p+ 2n = 2s + 2t. Subtracting 2n from 
both sides and factoring, we have m + p = 2s + 2t —2n = 
2(s + t — n). Because we have written m + p as 2 times 
an integer, we conclude that m + pis even. 7. Because n 
is odd, we can writen = 2k + 1 for some integer k. Then 
(k4+1)?—k? =k? 4+2k4+1—-k? =2k+1=n. 9. Suppose 
that r is rational and 7 is irrational and s = r +7 is rational. 
Then by Example 7, s+ (—r) = i is rational, which is a contra- 
diction. 11. Because /2-./2 = 2 is rational and \/2 is irra- 
tional, the product of two irrational numbers is not necessarily 
irrational. 13. Proof by contraposition: If 1 /x were rational, 
then by definition 1/x = p/q for some integers p and g with 
q # 0. Because 1/x cannot be 0 (if it were, then we’d have 
the contradiction | = x-0 by multiplying both sides by x), we 
know that p 4 0. Now x = 1/(1/x) = 1/(p/q) = q/p by the 
usual rules of algebra and arithmetic. Hence, x can be written 
as the quotient of two integers with the denominator nonzero. 
Thus by definition, x is rational. 15. Assume that it is not 
true thatx > lor y > 1.Thenx < 1 and y < 1. Adding these 
two inequalities, we obtain x + y < 2, which is the negation 
ofx+y>2. 17. a) Assume that n is odd, son = 2k+ 1 for 
some integer k. Then m+5= 2(4k3 +6k2+3k+3). Because 
n> + 5 is two times some integer, it is even. b) Suppose that 
n> + 5 is odd and n is odd. Because n is odd and the prod- 
uct of two odd numbers is odd, it follows that n2 is odd and 
then that n? is odd. But then 5 = (n> + 5) — n> would have 
to be even because it is the difference of two odd numbers. 
Therefore, the supposition that n3 + 5 and n were both odd is 
wrong. 19. The proposition is vacuously true because 0 is 
not a positive integer. Vacuous proof. 21. P(1) is true be- 
cause (a + b)! =at+b>a'!+b! =a +b. Direct proof. 
23. If we chose 9 or fewer days on each day of the week, this 
would account for at most 9 - 7 = 63 days. But we chose 64 
days. This contradiction shows that at least 10 of the days we 
chose must be on the same day of the week. 25. Suppose by 
way of contradiction that a/b is a rational root, where a and b 
are integers and this fraction is in lowest terms (that is, a and 
b have no common divisor greater than 1). Plug this proposed 
root into the equation to obtain a3/b*? + a/b + 1 = 0. Mul- 
tiply through by b? to obtain a? + ab? + b> = 0. If a and b 
are both odd, then the left-hand side is the sum of three odd 
numbers and therefore must be odd. If a is odd and b is even, 
then the left-hand side is odd + even + even, which is again 
odd. Similarly, if a is even and b is odd, then the left-hand 
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side is even + even + odd, which is again odd. Because the 
fraction a/b is in simplest terms, it cannot happen that both 
a and b are even. Thus in all cases, the left-hand side is odd, 
and therefore cannot equal 0. This contradiction shows that 
no such root exists. 27. First, assume that n is odd, so that 
n = 2k+1 for some integer k. Then 5n + 6 = 5(2k+1)+6= 
10k + 11 = 2(5k + 5) + 1. Hence, 5n + 6 is odd. To prove 
the converse, suppose that n is even, so that nm = 2k for some 
integer k. Then 5n + 6 = 10k + 6 = 2(5k + 3), so 5n + 6is 
even. Hence, n is odd if and only if 5n+6is odd. 29. This 
proposition is true. Suppose that m is neither 1 nor —1. Then 
mn has a factor m larger than 1. On the other hand, mn = 1, 
and | has no such factor. Hence, m = 1 orm = —1. In the 
first case n = 1, and in the second case n = —1, because 
n = 1/m. 31. We prove that all these are equivalent to x 
being even. If x is even, then x = 2k for some integer k. There- 
fore 3x +2 =3-2k+2 = 6k+2 = 2(3k +1), which is even, 
because it has been written in the form 2t, where t = 3k + 1. 
Similarly, x +5 = 2k+5 = 2k+4+4+1=2(k4+2)+1, 
so x + 5 is odd; and x2 = (2k)? 2(2k), so x2 
is even. For the converses, we will use a proof by contra- 
position. So assume that x is not even; thus x is odd and 
we can write x = 2k + 1 for some integer k. Then 
3x4+2 = 3(2k+1)+2 = 6k+5 = 2(3k+2)+1, which is odd 
(i.e., not even), because it has been written in the form 2t + 1, 
where t = 3k + 2. Similarly, x + 5 = 2k+1+5 = 2(k+3), 
so x + 5 is even (i.e., not odd). That x2 is odd was already 
proved in Example 1. 33. We give proofs by contraposition 
of (i) > (ii), (ii) > (i), @) — (ii), and (iii) > (7). 
For the first of these, suppose that 3x + 2 is rational, namely, 
equal to p/q for some integers p and g with g ~ 0. Then we 
can write x = ((p/q) — 2)/3 = (p — 2q)/(3q), where 
3q # 0. This shows that x is rational. For the second condi- 
tional statement, suppose that x is rational, namely, equal to 
p/q for some integers p and q with g 4 0. Then we can write 
3x+2 = (3p+2q)/q, where gq 4 0. This shows that 3x +2 is 
rational. For the third conditional statement, suppose that x /2 
is rational, namely, equal to p/q for some integers p and q 
with g 4 0. Then we can write x = 2p/q, where q 4 0. This 
shows that x is rational. And for the fourth conditional state- 
ment, suppose that x is rational, namely, equal to p/q for some 
integers p and q withg # 0. Then we can write x/2 = p/(2q), 
where 2g # O. This shows that x/2 is rational. 35. No 
37. Suppose that pi) > pa > p2 > ps > p3 —> pi. To 
prove that one of these propositions implies any of the others, 
just use hypothetical syllogism repeatedly. 39. We will give 
a proof by contradiction. Suppose that a}, a2, ..., dy are all 
less than A, where A is the average of these numbers. Then 
a, +a. +---+a, <nA. Dividing both sides by n shows 
that A = (a, +d. +---+a,)/n < A, which is a contradic- 
tion. 41. We will show that the four statements are equiv- 
alent by showing that (7) implies (ii), (ii) implies (iii), (iii) 
implies (iv), and (iv) implies (7). First, assume that n is even. 
Then n = 2k for some integer k. Thenn + 1 = 2k + 1, so 
n + 1 is odd. This shows that (i) implies (ii). Next, suppose 
that n + 1 is odd, son + 1 = 2k + 1 for some integer k. 
Then 3n +1 = 2n + (n+ 1) = 2(n + k) + 1, which 


shows that 3n + 1 is odd, showing that (ii) implies (iii). Next, 
suppose that 3n + 1 is odd, so 3n + 1 = 2k + 1 for some 
integer k. Then 3n = (2k + 1) — 1 = 2k, so 3n is even. 
This shows that (iii) implies (iv). Finally, suppose that n is 
not even. Then n is odd, son = 2k + 1 for some integer k. 
Then 3n = 3(2k+ 1) = 6k +3 = 2(3k+ 1)+1, so 3n is odd. 
This completes a proof by contraposition that (iv) implies (i). 


Section 1.8 


LP pl a=2S2=24P +1] se4S257 415 
10>8=23;44+1=17> 16=24 3.Ifx < y, 
then max(x, y) + min(x,y) = y+x=x+y.Ifx > y, 
then max(x, y) + min(x, y) = x + y. Because these are 
the only two cases, the equality always holds. 5. Because 
|x — y| = |y — x|, the values of x and y are interchange- 
able. Therefore, without loss of generality, we can assume that 
x > y. Then (x + y— (x — y))/2=@+y-—x+y)/2= 
2y/2 = y = min(x, y). Similarly, (x + y + (x — y))/2 = 
(«x +y+x- y)/2 = 2x/2 = x = max(x, y). 
7. There are four cases. Case 1: x > O and y > O. Then 
|x| + ly] = x+y = |x + y|. Case 2:x < Oandy < 0. 
Then |x| + |y| = —x + (~y) = —(«+ y) = |x + y| because 
x+y <0. Case 3:x > Oand y < 0.Then|x|+|y| =x+(—y). 
Ifx > —y, then |x + y| = x+y. But because y < 0,—y > y, 
so |x| +]y] =x+(-—y) >x+y=|x+y|. Ifx < —y, then 
jx+y| = —-—(x+y) = —x+(—y). But because x > 0,x > —x, 
so |x|+]y| =x + (—y) > —x + (—y) = [x+y]. Case 4: x <0 
and y > 0. Identical to Case 3 with the roles of x and y re- 
versed. 9. 10,001, 10,002,..., 10,100 are all nonsquares, 
because 1007 = 10,000 and 1017 = 10,201; constructive. 
11.8 = 2and9 = 3? 13.Letx = 2andy = V2. If 
x) =297 is irrational, we are done. If not, then let x = 2v? and 
y = V2/4. Then x” = (2V2)¥2/4 = gv2W2/4 — 91/2 = /9, 
15. a) This statement asserts the existence of x with a certain 
property. If we let y = x, then we see that P(x) is true. If y 
is anything other than x, then P(x) is not true. Thus, x is the 
unique element that makes P true. b) The first clause here 
says that there is an element that makes P true. The second 
clause says that whenever two elements both make P true, 
they are in fact the same element. Together these say that P 
is satisfied by exactly one element. c) This statement asserts 
the existence of an x that makes P true and has the further 
property that whenever we find an element that makes P true, 
that element is x. In other words, x is the unique element that 
makes P true. 17. The equation |a — c| = |b —c| is equiv- 
alent to the disjunction of two equations: a — c = b—c or 
a—c = —b-+c. The first of these is equivalent toa = b, 
which contradicts the assumptions made in this problem, so 
the original equation is equivalent toa —c = —b+c. By 
adding b + c to both sides and dividing by 2, we see that this 
equation is equivalent toc = (a + b)/2. Thus, there is a 


IV 


unique solution. Furthermore, this c is an integer, because the 
sum of the odd integers a and bis even. 19. We are being 
asked to solve n = (k — 2) + (k + 3) for k. Using the usual, 
reversible, rules of algebra, we see that this equation is equiv- 
alent to k = (n — 1)/2. In other words, this is the one and 
only value of k that makes our equation true. Because n is odd, 
n—1liseven,sokisaninteger. 21. If x is itself an integer, 
then we can take n = x andeé = (QO. No other solution is 
possible in this case, because if the integer n is greater than x, 
then n is at least x + 1, which would make € > 1. If x is not 
an integer, then round it up to the next integer, and call that 
integer n. Lete = n — x. Clearly 0 < € < 1; this is the 
only ¢€ that will work with this n, and n cannot be any larger, 
because € is constrained to be less than1. 23. The harmonic 
mean of distinct positive real numbers x and y is always less 
than their geometric mean. To prove 2xy/(x + y) < /xy, 
multiply both sides by (x + y)/(2,/xy) to obtain the equiv- 
alent inequality ./xy < (x + y)/2, which is proved in Ex- 
ample 14. 25. The parity (oddness or evenness) of the sum 
of the numbers written on the board never changes, because 
j + and |j — k| have the same parity (and at each step we 
reduce the sum by j + & but increase it by |j — k|). There- 
fore the integer at the end of the process must have the same 
parity as 1+ 2+.---+ (2n) = n(2n + 1), which is odd 
because n is odd. 27. Without loss of generality we can as- 
sume that n is nonnegative, because the fourth power of an 
integer and the fourth power of its negative are the same. We 
divide an arbitrary positive integer n by 10, obtaining a quo- 
tient k and remainder /, whence n = 10k + J, and / is an 
integer between 0 and 9, inclusive. Then we compute n* in 
each of these 10 cases. We get the following values, where X 
is some integer that is a multiple of 10, whose exact value we 
do not care about. (10k + 0)* = 10,000k4 = 10,000k* + 0, 
(10k + 1)4 = 10,000k4 4+ X-R+X-R4+X-k4+1, 
(10k + 2)* = 10,000k4 + X-K+X-k4+X-k+4+ 16, 
(10k + 3) = 10,000k4 + X-k2 + X-k24+X-k4+81, 
(10k + 4)* = 10,000k* + X- 74+ X-k24+X-k + 256, 
(10k + 5)* = 10,000k* + X-k74+X-k2 +X -k + 625, 
(10k + 6)4 = 10,000k* + X-k7>4+X-k7 4+ X-k +1296, 
(10k + 7)4 = 10,000k4 + X-k?> 4+ X-k? + X-k +2401, 
(10k + 8)* = 10,000k* + X-k7> + X-k27 + X-k + 4096, 
(10k + 9)* = 10,000k4 + X -k7> +X +k? + X-k + 6561. 
Because each coefficient indicated by X is a multiple of 10, 
the corresponding term has no effect on the ones digit of the 
answer. Therefore the ones digits are 0, 1, 6, 1, 6, 5, 6, 1, 6, 
1, respectively, so it is always a0, 1,5, o0r6. 29. Because 
n> > 100 foralln > 4, we need only note thatn = 1, 
n = 2,n = 3,andn = 4 do not satisfy n2+n3 = 100. 
31. Because 54 = 625, both x and y must be less than 5. 
Then x* + y* < 44444 = 512 < 625. 33. If it is not 
true thata < Yn, b < Yn, orc < Yn, thena > Yn, 
b > Yn, andc > Yn. Multiplying these inequalities of 
positive numbers together we obtain abe < (i/n)? = n, 
which implies the negation of our hypothesis that n = abc. 
35. By finding a common denominator, we can assume that 
the given rational numbers are a/b and c/b, where b is a pos- 
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itive integer and a and c are integers witha < c. In particular, 
(a+1)/b < c/b. Thus, x = (a+ 5V2)/b is between the two 
given rational numbers, because 0 < V2 <2. Furthermore, x 
is irrational, because if x were rational, then 2(bx — a) = J2 
would be as well, in violation of Example 10 in Section 1.7. 
37. a) Without loss of generality, we can assume that the x 
sequence is already sorted into nondecreasing order, because 
we can relabel the indices. There are only a finite number of 
possible orderings for the y sequence, so if we can show that 
we can increase the sum (or at least keep it the same) when- 
ever we find y; and y; that are out of order (i.e.,i < j but 
yi > yj) by switching them, then we will have shown that 
the sum is largest when the y sequence is in nondecreasing 
order. Indeed, if we perform the swap, then we have added 
xiyj + x;y; to the sum and subtracted x;y; + xjy;. The 
net effect is to have added xjyj + xjy¥i — XiVi — XV] = 
(xj; — xi)(yi — yj), which is nonnegative by our ordering as- 
sumptions. b) Similar to part(a) 39.a)6 > 3 > 10 > 
5> 16>8-~4->2-1b7> 22> 11> 34> 
17 > 52 > 26 > 13 > 40 > 20 —- 10 >5-> 16> 
87> 4->2->1 oO17> 52> 2- 13> 40 > 
20-10 ~ 5 > 16> 8-7>4-7>2-> 1 
d) 21 > 64-32 - 16> 8 ~ 452-1 41. Without 
loss of generality, assume that the upper left and upper right 
corners of the board are removed. Place three dominoes hor- 
izontally to fill the remaining portion of the first row, and fill 
each of the other seven rows with four horizontal dominoes. 
43. Because there is an even number of squares in all, either 
there is an even number of squares in each row or there is an 
even number of squares in each column. In the former case, 
tile the board in the obvious way by placing the dominoes hor- 
izontally, and in the latter case, tile the board in the obvious 
way by placing the dominoes vertically. 45. We can rotate 
the board if necessary to make the removed squares be | and 
16. Square 2 must be covered by a domino. If that domino is 
placed to cover squares 2 and 6, then the following domino 
placements are forced in succession: 5-9, 13-14, and 10-11, 
at which point there is no way to cover square 15. Otherwise, 
square 2 must be covered by a domino placed at 2-3. Then 
the following domino placements are forced: 4-8, 11-12, 6-7, 
5-9, and 10-14, and again there is no way to cover square 15. 
47. Remove the two black squares adjacent to a white corner, 
and remove two white squares other than that corner. Then no 
domino can cover that white corner. 


49. a) 


(1) (2) (3) (4) (5) 
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b) The picture shows tilings for the first four patterns. 
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To show that pattern 5 cannot tile the checkerboard, label the 
squares from | to 64, one row at a time from the top, from left 
to right in each row. Thus, square 1 is the upper left corner, 
and square 64 is the lower right. Suppose we did have a tiling. 
By symmetry and without loss of generality, we may suppose 
that the tile is positioned in the upper left corner, covering 
squares 1, 2, 10, and 11. This forces a tile to be adjacent to 
it on the right, covering squares 3, 4, 12, and 13. Continue in 
this manner and we are forced to have a tile covering squares 
6, 7, 15, and 16. This makes it impossible to cover square 8. 
Thus, no tiling is possible. 


Supplementary Exercises 


l.a)g—>p b)qAp o)-qV-p d)q<p_ 3.a)The 
proposition cannot be false unless —p is false, so p is true. If 
Pp is true and q is true, then —q A (p — 4q) is false, so the 
conditional statement is true. If p is true and q is false, then 
Pp — ¢q is false, so ag A (p — q) is false and the conditional 
statement is true. b) The proposition cannot be false unless 
q is false. If g is false and p is true, then (p V q) A —p is 
false, and the conditional statement is true. If g is false and p 
is false, then (p V gq) A =p is false, and the conditional state- 
mentistrue. 5.7-q > 7p; p—>q;7p—>-q_ 7.(pa 
qArnr7rs)V(pAngA-7r A S)V (pPAn7Aq ArAS)V 
(-p Aq ArA-s)_ 9.Translating these statements into 
symbols, using the obvious letters, we have -tf — -7g8, 

1g —> -q,r — gq, and —t A r. Assume the state- 
ments are consistent. The fourth statement tells us that —+¢ 
must be true. Therefore by modus ponens with the first state- 
ment, we know that —¢ is true, hence (from the second state- 
ment), that —q is true. Also, the fourth statement tells us 
that r must be true, and so again modus ponens (third state- 
ment) makes gq true. This is a contradiction: g A —q. Thus 
the statements are inconsistent. 11. Reject-accept-reject- 
accept, accept-accept-accept-accept, accept-accept-reject- 
accept, reject-reject-reject-reject, reject-reject-accept-reject, 
and reject-accept-accept-accept 13. Aaron is a knave and 
Crystal is a knight; it cannot be determined what Bohan is. 
15. Brenda 17. The premises cannot both be true, because 


they are contradictory. Therefore it is (vacuously) true that 
whenever all the premises are true, the conclusion is also 
true, which by definition makes this a valid argument. Be- 
cause the premises are not both true, we cannot conclude 
that the conclusion is true. 19. Use the same propositions 
as were given in Section 1.3 for a 9 x 9 Sudoku puzzle, 
with the variables indexed from | to 16, instead of from 1 
to 9, and with a similar change for the propositions for the 
4 x 4 blocks: Ae Te PY ed 1 it bs iVj , p(4r +i, 4s + 
j,n). 21.a)F b)T c)F d)T e)F f)T 23. Many an- 
swers are possible. One example is United States senators. 
25. Vxdydz (y 4 zAVwW(P(w,x) e— wW=yVvw = 2Z))) 
27.a) —5dx P(x) b)Ax(P(x) A Vy(P(y) > y = x)) 
c) Axydx2(P(x1) A P(xr2) A xy A x2 AVY(P(y) > (Y= 
xiVy =Xx2))) d) dxy 4x25 13(P (x1 )AP (42) AP (x3)Ax1 # 
2A xX AF xX A XQ FF x3 A Wy(P(y) > 
(y = x V y=x2V y = 4x3))) 29. Suppose that 
dx(P(x) > Q(x)) is true. Then either Q(xo) is true for 
some Xo, in which case Vx P(x) — Ax Q(x) is true; or P(x9) 
is false for some xg, in which case Vx P(x) > 4x Q(x) is 
true. Conversely, suppose that dx(P(x) > Q(x)) is false. 
That means that Vx(P(x) A —Q(x)) is true, which implies 
Vx P(x) and Vx(=Q(x)). This latter proposition is equivalent 
to 74x Q(x). Thus, Vx P(x) > 4x Q(x) is false. 31. No 
33. Vx Vz dy T(x, y, z), where T(x, y, z) is the statement 
that student x has taken class y in department z, where 
the domains are the set of students in the class, the set of 
courses at this university, and the set of departments in the 
school of mathematical sciences 35. 4!xd!y T(x, y) and 
AxVz((AyVw(T (z, w) = w = y)) + z =x), where T(x, y) 
means that student x has taken class y and the domain is all 
students in this class 37. P(a) + Q(a) and Q(a) > R(a) 
by universal instantiation; then ~Q(a) by modus tollens and 
—P(a) by modustollens 39. We give a proof by contraposi- 
tion and show that if ./x is rational, then x is rational, assuming 
throughout that x > 0. Suppose that ./x = p/q is rational, 
q #0. Then x = (./x)* = p?/q? is also rational (q? is again 
nonzero). 41. We can give a constructive proof by letting 
m = 10°41. Then m2 = (10° + 1)2 > (105)? = 191000, 
43. 23 cannot be written as the sum of eight cubes. 45. 223 
cannot be written as the sum of 36 fifth powers. 


CHAPTER 2 


Section 2.1 


1. a) {—1,1}  b) {1,2,3,4,5,6,7,8,9,10,11} ce) {0,1,4, 9, 16, 
25, 36, 49, 64,81} d)W 3. a) The first is a subset of the 
second, but the second is not a subset of the first. b) Neither 
is a subset of the other. c) The first is a subset of the sec- 


ond, but the second is not a subset of the first. 5. a) Yes 
b) No c)No_ 7.a) Yes b)No c) Yes d) No e)No f) No 
9. a) False b) Falsec) Falsed) Truee) False f) False g) True 
1l.a) True b) True c) False d) True e) True f) False 


13, 


Januar 
February 


March April 


October 


September 


November 
e December 


15. The dots in certain regions indicate that those regions are 
not empty. 


17. Suppose that x € A. Because A C B, this implies that 
x € 8B. Because B CC, we see that x € C. Because 
x € A implies that x € C, it follows that ACC. 19.a)1 
b)1 c)2 d)3 21. a) {H, {a}} b) {O, {a}, {b}, {a, b}} 
) (9, {0}, {0}, (9, (}}} @BBa) 8 by 16 c)2 25. For 
the “if” part, given A C 8B, we want to show that that 
P(A) © P(B), ie., if C C Athen C C B. But this fol- 
lows directly from Exercise 17. For the “only if” part, given 
that P(A) C P(B), we want to show that A C B. Suppose 
a € A. Then {a} C A, so {a} € P(A). Since P(A) C P(B), 
it follows that {a} € P(B), which means that {a} C B. 
But this implies a € B, as desired. 27. a) {(a, y), (b, y), 
(c, y), , y), (a, z), (B, 2), (c, 2), (d, Z)} BD) {, a), (y, 4), 
(y, c), (y, a), (Z, a), (Z, Bb), (Zz, €), (Zz, d)} 29. The set of 
triples (a, b, c), where a is an airline and b and c are cities. 
A useful subset of this set is the set of triples (a, b, c) for 
which a flies between bandc. 31.0x A= {(x,y)|xe€ 
Jandy € A}=BO={(x,y)|xeAandye VJ} =AxG 
33. a) {(0, 0), (0, 1), (0, 3), A, 0), C1, 1), C1, 3), (3, 0), (3, 1), 
(3, 3)} b) {d, D, C, 2), CU, a), A, ), 2, 1), 2, 2), 2, a), 
(2, b), (a, 1), (a, 2), (a, a), (a, b), (b, 1), (b, 2), (b, a), (b, b)} 
35.mn 37. m" 39. The elements of A x B x C consist 
of 3-tuples (a, b,c), where a € A,b € B, andc € C, whereas 
the elements of (A x B) x C look like ((a, b), c)—ordered 
pairs, the first coordinate of which is again an ordered pair. 


every integer is positive. False d) There is areal number equal 
(Giitslownl square Tie 3. a) (—1, 0, 1} b)Z—{0, 1} OD 
45. We must show that {{a}, {a, b}} = {{c}, {c, d}} if and 
only ifa = cand b = d. The “if” part is immediate. So 
assume these two sets are equal. First, consider the case when 
a #b. Then {{a}, {a, b}} contains exactly two elements, one 
of which contains one element. Thus, {{c}, {c, d}} must have 
the same property, so c € d and {c} is the element containing 
exactly one element. Hence, {a} = {c}, which implies that 
a = c. Also, the two-element sets {a, b} and {c, d} must be 
equal. Because a = c anda ¥< b, it follows that b = d. 
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Second, suppose that a = b. Then {{a}, {a, b}} = {{a}}, a set 
with one element. Hence, {{c}, {c, d}} has only one element, 
which can happen only when c = d, and the set is {{c}}. It then 
follows thata =candb=d. 47. Let S = {a,a2,..., dy}. 
Represent each subset of S with a bit string of length n, where 
the ith bit is 1 if and only if a; € S. To generate all subsets of 
S, list all 2” bit strings of length n (for instance, in increasing 
order), and write down the corresponding subsets. 


Section 2.2 


gir he set of students who live within one mile of school 
and walk to classes b) The set of students who live within 
one mile of school or walk to classes (or do both) c) The 
set of students who live within one mile of school but 
do not walk to classes d)The set of students who walk 
to classes but live more than one mile away from school 
@h) {0,1,2,3,4,5,6} b) {3} c) {1, 2,4,5} d) {0,6} 5.A= 
[-GeAf=(elGe € A} =a |x € APSA 
TayAUU={x|xE AVxEeUS={x|xEe AV T= 
{x | T}=UbD ANGDB={x|x EAAxEBD=H= {x |xeE 
AAF}={x|F}=9% 9.a) AUA={x|xe AVx ¢gA}=U 
bANA = {x|x € AAx ¢ A} =D Ila AU 
B= {x|xeAvxe BJ={x|xeBvxEeAJ=BUA 
DAN B= {x|xEeAAxe By={x|xe BaAxe A= 
BOA 13.Suppose x € AN (AU B). Thenx € A and 
x € AUB by the definition of intersection. Because x € A, 
we have proved that the left-hand side is a subset of the right- 
hand side. Conversely, let x € A. Then by the definition of 
union, x € A U B as well. Therefore x € AM (AU B) 
by the definition of intersection, so the right-hand side is 
a subset of the left-hand side. 1S.a)x € AUB 

x GAUB= 7 E€AVxE B)=7(x € A) AAV E B) 


xXxZAAXEBS=XEAAXEB=XEANB 


eax EANBNC=H=x¢ANBNC=x ¢ZAVXE 
BVx€C=xEAVxeBvxEeC=xEeAUBUC 


” ae clananc|ananc|a|B|c|auBuc 
i ae a | 1 0 0/};0)}0 0 
1 1 0 0 1 0 1 
1 0 1 0 1 0 1 
1 0 0 0 1 0 1 
Oo 1 1 0 1 1 1 
0 1 0 0 1 1 1 
0 0 1 0 1 1 1 
0 0 0 0 1 1 1 
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19. a) Both sides equal {x |x € AAx ¢ B}. b) A= ANU = 
AN(BUB)=(ANB)U(ANB) 21.x € AU(BUC) 
(x € AVV(x € (BUC) = WEADV(XE BVxX 
C) = XWEAVXEBV(XEC)=xE(AUB)U 
23.xE€AU(BNC) = WEA) V (XE(BN C)) 
(x € A) V (XR EE BAx€EC) B= W’EAVXE 
B) A (KE AVXEC)HzEX EC (AUBN(AUC) 
25. a) {4,6} b) {0,1,2,3,4,5,6,7,8,9, 10} c) {4, 5, 6, 8, 10} 
d) {0,2,4, 5,6,7,8,9,10} 27. a) The double-shaded portion 
is the desired set. 


Il Am ll 


A B 


Cc 


b) The desired set is the entire shaded portion. 


A B 


(a 


c) The desired set is the entire shaded portion. 


A B 


c 


29.a)BCA bD)ACB co) ANB=E d) Nothing, because 
thisis alwaystrue e)A=B 31.AC B=Vx(xeAnm 


xe B) = Vx(x €Bo>x¢A) = Vx(x € BO x 
A) = B C A 33.The set of students who are computer 
science majors but not mathematics majors or who are math- 
ematics majors but not computer science majors 35. An 
element is in (A U B) — (Af B) if it is in the union of A 
and B but not in the intersection of A and B, which means 
that it is in either A or B but not in both A and B. This is 
exactly what it means for an element to belong to A @ B. 
37.a) A®A = (A-—A)U(A—A) = OUBH= BH 
b) A@¥Y = (A-B)UW-—A) = AUB =A QDAGBUH= 
(A—U)U(U-—A)=GUA=A ADA@MAH=(A-A)VU 
(A—A) = AUA=U 39.B=G 41.Yes 43. Yes 
45. If AUB were finite, then it would have n elements for some 
natural number n. But A already has more than n elements, 
because it is infinite, and AU B has all the elements that A has, 
so AU B has more than n elements. This contradiction shows 
that AU B must be infinite. 47.a) {1,2,3,...,n} b) {1} 
49.a) A, b) {0,1} 5l.a) Z, {-1,0, 1} b)Z — {0}, 0 
co) R,[-1,1) d[l,o),4 53.a) {1, 2, 3, 4, 7, 8, 9, 10} 
b) {2, 4,5, 6, 7} c) {1, 10} 55. The bit in the ith position of 
the bit string of the difference of two sets is | if the ith bit 
of the first string is 1 and the ith bit of the second string is 0, 
and is O otherwise. 57.a) 11 1110 0000 0000 0000 0000 
0000 v 01 1100 1000 0000 0100 0101 0000 = 11 1110 1000 
0000 0100 0101 0000, representing {a, b, c, d, e, g, p, t, v} 


b) 11 1110 0000 0000 0000 0000 0000 A 01 1100 1000 0000 
0100 0101 0000 = 01 1100 0000 0000 0000 0000 0000, 
representing {b, c,d} ec) (11 1110 0000 0000 0000 0000 
0000 v 00 0110 0110 0001 1000 0110 0110) A (O1 1100 
1000 0000 0100 0101 0000 v 00 1010 0010 0000 1000 0010 
0111) = 11 1110 0110 0001 1000 0110 0110 A O1 1110 
1010 0000 1100 0111 0111 =01 11100010 0000 1000 0110 
0110, representing {b, c, d, e, i, o, t, u, x, y} d)11 1110 
0000 0000 0000 0000 0000 v 01 1100 1000 0000 0100 0101 
0000 v 00 1010 0010 0000 1000 0010 0111 v 00 0110 0110 
0001 1000 0110 0110 = 11 1110 1110 0001 1100 O111 
0111, representing {a,b,c,d,e,g,h,i,n,o,p,t,u,v,x,y,Z} 
59. a) {1, 2, 3, {1, 2, 3}} b) {(} ©) {H, {A} ad, {H}, 
{O, {O}}} 6l.a){3-a, 3-b, 1-c, 4-d} b){2-a, 2-b} 
ce) {l-a, 1-c} d){1-b, 4-d} e) {5-a, 5-b, 1-c,4-d} 
63.F = {0.4 Alice, 0.1 Brian, 0.6 Fred, 0.9 Oscar, 0.5 Rita}, 
R= {0.6 Alice, 0.2 Brian, 0.8 Fred, 0.1 Oscar, 0.3 Rita} 
65. {0.4 Alice, 0.8 Brian, 0.2 Fred, 0.1 Oscar, 0.5 Rita} 


Section 2.3 


1. a) f(O) is not defined. b) f(x) is not defined for x < 0. 
c) f(x) is not well-defined because there are two distinct 
values assigned to each x. 3.a) Nota function b)A func- 
tion c) Nota function 5.a) Domain the set of bit strings; 
range the set of integers b) Domain the set of bit strings; 
range the set of even nonnegative integers c) Domain the 
set of bit strings; range the set of nonnegative integers not 
exceeding 7 d) Domain the set of positive integers; range 
the set of squares of positive integers = {1, 4, 9, 16, ...} 
7.a) Domain Z* x Z*; range Zt b) Domain Z*; range 
{0, 1, 2, 3, 4, 5, 6, 7, 8, 9} c) Domain the set of bit strings; 
range N d) Domain the set of bit strings; range N 9. a) 1 
b)O c)O d)—1 e)3 f)—1 g)2 h)1_ 11. Only the 
function in part (a) 13. Only the functions in parts (a) and 
(d) 15.a) Onto b) Not onto c) Onto d) Not onto e) Onto 
17. a) Depends on whether teachers share offices b) One- 
to-one assuming only one teacher per bus c) Most likely not 
one-to-one, especially if salary is set by a collective bargain- 
ing agreement d) One-to-one 19. Answers will vary. a) Set 
of offices at the school; probably not onto b) Set of buses 
going on the trip; onto, assuming every bus gets a teacher 
chaperone c) Set of real numbers; not onto d) Set of strings 
of nine digits with hyphens after third and fifth digits; not 
onto 21. a) The function f(x) with f(x) = 3x + 1 when 
x > Oand f(x) = —3x +2 whenx <0 b) f(x) =|x|+1 
c) The function f(x) with f(x) = 2x + 1 when x > 0 and 
f(x) = —2x when x <0 d) f(x) = x*+4+1  23.a) Yes 
b) No c) Yes d)No_ 25. Suppose that f is strictly decreas- 
ing. This means that f(x) > f(y) whenever x < y. To 
show that g is strictly increasing, suppose that x < y. Then 
g(x) = 1/f(x) < 1/f(y) = g(). Conversely, suppose that g 
is strictly increasing. This means that g(x) < g(y) whenever 
x < y. To show that f is strictly decreasing, suppose that 
x < y. Then f(x) = 1/g(x) > 1/g(y) = f(y). 27. a) Let 
f be a given strictly decreasing function from R to itself. If 


a < b,then f(a) > f(b); ifa > b, then f(a) < f(b). 
Thus if a 4 b, then f(a) # f(b). b) Answers will vary; for 
example, f(x) = 0 for x < Oand f(x) = —x forx > 0. 
29. The function is not one-to-one, so it is not invertible. On 
the restricted domain, the function is the identity function on 
the nonnegative real numbers, f(x) = x, so it is its own in- 
verse. 3l.a) f(S) = {0, 1,3} b) f(S) = {0, 1, 3,5, 8} 
c) f(S) = {0, 8, 16, 40} d) f(S) = {1, 12, 33, 65} 
33. a) Let x and y be distinct elements of A. Because g is one- 
to-one, g(x) and g(y) are distinct elements of B. Because f is 
one-to-one, f(g(x)) = (f° g)(x) and f(g(y)) = (f og)(y) 
are distinct elements of C. Hence, f o g is one-to-one. b) Let 
y € C. Because f is onto, y = f(b) forsome b ¢€ B. 
Now because g is onto, b = g(x) for some x € A. Hence, 
y= f(b) = f(g(x)) = (fog) (x). It follows that fog is onto. 
35. No. For example, suppose that A = {a}, B = {b,c}, and 
C = {d}. Let g(a) = b, f(b) =d,and f(c) = d. Then f and 
f og are onto, but gisnot. 37. (f+ g)(x) =x? +x43, 
(fg)(x) = x3 + 2x2 +x+2 39. f is one-to-one because 
Ff (x1) = f(x2) > ax) +b =ax2+b > ax, =ax2 > XH = 
x2. f is onto because f ((y —b)/a) = y.f~!(y) =(y —b)/a. 
41.aA=B=R,S={x|x>O0},T = {x |x < 0}, 
f@wa= x” b) It suffices to show that FISAF(L) © f(SOT). 
Let y € B be an element of f(S)M f(T). Then y € f(S), 
so y = f(x1) for some x; € S. Similarly, y = f(x2) 
for some x2 € T. Because f is one-to-one, it follows that 
xX; = x2. Thereforex; € SON T,soy € f(SOT). 
43.a){x | 0 < x < 1} bD){x | -1 < x < 2} OD 
45. f-'(S) = {x € A| f(x) ¢S} = {xE Al f@ES) 
= f—'(S) 47. Let x = |x| + €, where € is a real number 
with 0 < « < 1. Ife < 5, then |x] —-1 <x—} < |x], s0 
[x — 5] = |x| and this is the integer closest to x. Ife > 5 
then |x] <x — 4 < [x] + 1,80 [x — 5] = [x] + 1and 
this is the integer closest to x. Ife = 55 then [x — 5] = [x], 
which is the smaller of the two integers that surround x and 
are the same distance from x. 49. Write the real number x 
as |x| + €, where € is areal number with 0 < € < 1. Because 
€ = x — |x], it follows thatO < —|[x] < 1. The first two 
inequalities, x —1 < |x] and |x] < x, follow directly. For the 
other two inequalities, write x = [x] — €’, where 0 < €’ < 1. 
Then 0 < [x] —x < 1, and the desired inequality follows. 
51. a) If x < n, because |x| < x, it follows that [x] < n. 
Suppose that x > n. By the definition of the floor function, it 
follows that |x| >. This means that if |x| <n, thenx <n. 
b) Ifn < x, then because x < [x], it follows thatn < [x]. 
Suppose that n > x. By the definition of the ceiling function, 
it follows that [x] < n. This means that ifn < [x], then 
n< x. 53.Ifn is even, thenn = 2k for some integer k. 
Thus, [n/2] = [k] =k = n/2. If n is odd, thenn = 2k + 1 
for some integer k. Thus, [1/2] = [A+ 5] =k=(n-—1)/2. 
55. Assume that x > 0. The left-hand side is [—x] and the 
right-hand side is —|x]. If x is an integer, then both sides 


equal —x. Otherwise, let x = n + €, where n is a natu- 
ral number and ¢€ is a real number with O < e€ < 1. Then 
[—x] = [—-n — €] = -—n and -—|x] = -—|n+e] = —-n 


also. When x < 0, the equation also holds because it can 
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be obtained by substituting —x for x. 57. [b] — La] -— 1 
59. a) 1 b)3 c) 126 d)3600 61. a) 100 b) 256 c) 1030 
d) 30,200 


63. 4+ oo 
3 oo 
2 oo 
1+ eo 
o—O- + 
—2 -1 O 1 2 3 4 
eo-l 
eo +-2 
65. 3+ —o 
at 
1 —o 
—o 
2 --1 0 1 2 3 
-l 
—) -2 
—" 3 
67. a) : = 
2+ — 
1l+e—o 
+—}-2=-0-+ + 
= a) 2 4 2 
—o+-1 
— +2 
a 3 
c) 3 — d) 4 7° 
ee 3 — 
ice 2 __ 
t —— : 
-12-9 -6 -3 ‘a 6 9 12 ~——_+ ++ +—+—_+—_ +++ 
-1 a 1 
— —2 
_ —2 
oe =3 ~e 3 
e) 4 ° f) —_— s+ 
3 é o e 44 e 
ot . 34 _— 
2+ — 
1o—o 
1+ eo 
py} 4 7 4 4 
2 =| 1 2 i a a e 
—— 2 -1 |, 1 2 
. pie) 2 
—o =a 3 
° —4 -4 


g) See part (a). 69. f-!(y) =(y—1)!? 71. a) fana(x) = 
loexeEANBexc Aandxe Bs fa(x) = 1 and 
fa(x) = 1 <> fax) fa) = 1 b) fausee) =loxe 
AUB =x € Aorx € Bs fa(xe) = 1 
or fa(x) = 1 <_ fate) + fae) — fa) fa) =1 
oe) fya)HloxeAox As fax) =0o1-fa(x)= 
1 d@) faoen(x) =lox Ee APB (we Aandx ¢ B)or 
(x ¢ Aandx € B) > f(x) + fa(x) — 2fa(x) fa(x) = 1 
73. a) True; because |x] is already an integer, [|x]] = Lx]. 
b) False; x = 5 is acounterexample. c) True; if x or y is an 
integer, then by property 4b in Table 1, the difference is 0. If 
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neither x nor y is an integer, then x =n+eandy=m-+6, 
where n and m are integers and € and 6 are positive real num- 
bers less than 1. Thenm +n <x+y <m-+n+2,s0 
[x+y] is eitherm-+n-+1 orm+n-+2. Therefore, the given 
expression is either (1 + 1) +(m+1)—(m+n+1)=1lor 
(n+1)+(m+1) -—(m+n-+2) =0, as desired. d) False; 
x= i and y = 3 is acounterexample. e) False; x = 5 is 
acounterexample. 75. a) If x is a positive integer, then the 
two sides are equal. So suppose that x = n* + m+ €, where 
n? is the largest perfect square less than x, m is a nonnegative 
integer, and 0 < € < 1. Then both /x and /[x] = /n? +m 
are between n and n + 1, so both sides equal n. b) If x isa 
positive integer, then the two sides are equal. So suppose that 
x = n? —m —e, where n? is the smallest perfect square 
greater than x, m is a nonnegative integer, and € is a real num- 
ber with 0 < € < 1. Then both /x and /Jx] = V/n2—m 
are between n — | and n. Therefore, both sides of the equa- 
tionequaln. 77. a) Domain is Z; codomain is R; domain of 
definition is the set of nonzero integers; the set of values for 
which f is undefined is {0}; not a total function. b) Domain 
is Z; codomain is Z; domain of definition is Z; set of values 
for which f is undefined is ; total function. c) Domain is 
Z x Z; codomain is Q; domain of definition is Z x (Z — {0}); 
set of values for which f is undefined is Z x {0}; not a total 
function. d) Domain is Z x Z; codomain is Z; domain of 
definition is Z x Z; set of values for which f is undefined 
is %; total function. e) Domain is Z x Z; codomain is Z; 
domain of definitions is {(m,n) | m > n}; set of values 
for which f is undefined is {(m,n) | m < n}; not a total 
function. 79. a) By definition, to say that S has cardinality 
m is to say that S has exactly m distinct elements. Therefore 
we can assign the first object to 1, the second to 2, and so on. 
This provides the one-to-one correspondence. b) By part (a), 
there is a bijection f from S to {1,2,...,m} and a bijection 
g from T to {1,2,...,m}. Then the composition g~! o f is 
the desired bijection from S to T. 


Section 2.4 

l.a)3 b)—1 c)787 d) 2639 3.a)ag = 2,a, = 3, 
a2 =5, a3 =9 b) ag = 1, a, =4, ag =27, 3 256 
Cc) ao 0, a1 0, a2 1, a3 1 da =0,q = 1, 


a2 = 2,a3 = 3 5.a)2,5, 8, 11, 14, 17, 20, 23, 26, 29 
b) 1, 1, 1, 2, 2, 2, 3, 3, 3,4 ©) 1, 1, 3, 3,5, 5, 7, 7, 9, 9 
d)—1, —2, —2, 8, 88, 656, 4912, 40064, 362368, 
3627776 =e) 3, 6, 12, 24, 48, 96, 192, 384, 768, 1536 
f) 2,4, 6, 10, 16, 26, 42, 68, 110, 178 g) 1, 2, 2, 3,3, 3, 3,4, 
4,4 h)3, 3,5, 4,4, 3,5,5,4,3 7. Each term could be 
twice the previous term; the nth term could be obtained from 
the previous term by adding n — 1; the terms could be the 
positive integers that are not multiples of 3; there are in- 
finitely many other possibilities. 9. a) 2, 12,72, 432, 2592 
b) 2, 4, 16, 256, 65,536c¢) 1,2,5,11,26 d)1, 1,6, 27, 204 


e)1,2,0,1,3 11. a)6, 17, 49, 143, 421 b) 49 = 
5-17 -—6.- 6 143 = 5-49 —6-.- 17,421 = 
5-143 —6-49  c)Sady,_1 — 6a,  =5(2""' +5- 


BPN) es: GORE A Ba BP 8y 2"-2(10 — 6) + 
3"-2(75 — 30) = 27-2. 4 + 3%-2.9.5=2" 43" - SH ay 
13. a) Yes b)No c)No d) Yes e) Yes f) Yes g) No h) No 
15.a)dj—-1 + 2a,-2 +2n —9 = —(n 1) 2+2 
[-(@ — 2) +2] + 2n —9 = -n4+2 = aq D)agy_-1 + 
2dan—2 + 2n — 9 = 5(—1)""! — (n — 1) +242[5(-1)"? — 
(n — 2) +2] + 2n —9=5(-1)"~ 2(-1 +2) —n+2=ay 
C) An—1 + 2ap_2 + 2n—9 = 3(—1)""!4-2"-! — @—1) +24 
2[3(—1)"-2_ + 2"-2 — (n—2) + 2] + 2n-9 = 3(-1)""? 
(-1 + 2) + 2702 4+ 2)-n +2 = a Gay +t 
Qdn_2 +2n—9 = 7-2"-! — (n—1) + 2 + 27-2"? - 
(n — 2) + 2]+2n—9 = 2"-2(7.242-7)—n +2=a 
17. a) a, =2-3" b)ay =2n+3 Cha, =14+n(n+1)/2 
da, = n2+4n+4 ea, =1 fog, = B"t!-—1)/2 
2) dy, = Sn! h) ay =2"n! 19. a) dy =3an_1 b) 5,904,900 
2laa, = n + @-1,4 = 0 Day = 78 
cC) da, =n(n+1)/2 23. B(k) = [1 + (0.07/12) ] Bk — 1) — 
100, with B(O) = 5000 25. a) One | and one 0, followed 
by two Is and two Os, followed by three 1s and three 0s, 
and so on; 1, 1, 1 b) The positive integers are listed in in- 
creasing order with each even positive integer listed twice; 
9, 10, 10. c) The terms in odd-numbered locations are the 
successive powers of 2; the terms in even-numbered loca- 
tions are all 0; 32, 0, 64. d) a, =3-2"~!; 384, 768, 1536 
e)a, = 15 T(n 1) = 22 — 7n; —34, —41, —48 
f)dy) = (n? +n + 4)/2; 57, 68, 80 g)an = 2n?; 
1024, 1458, 2000 h)a, = n! + 1; 362881, 3628801, 
39916801 27. Among the integers 1, 2,..., adn, where ay 
is the nth positive integer not a perfect square, the nonsquares 
are a|,da2,...,dy and the squares are 17,22, Ses Jk, where k 
is the integer with k*> <n+k < (k + 1)*. Consequently, 
dn =n-+k, where R< Gn < (k+ 1)?. To find k, first note 
thatk? <n+k <(k+1)?,sok7+1 <n+k < (k+1?-1. 
Hence, (k—5)°+3 = —k+1 <n <k?+k = (k+5)°-3. 
It follows thatk — 3 < Jn < k+5,80k = {Vn} 
andd,; =n+k =n+{Jn}. 29.a)20 b)11 c)30 
d)511 31.a) 1533 b)510 c) 4923 d) 9842 33.a)21 
b)78 c) 18 d) 18 35. Via Gj — aj-1) = a — a 
37.a)n? b)n(n + 1)/2 39.15150 41, Setar) 4 
Mn+ + (n+ 1)(m — (n+ 1)? + 1), where n = |,/m] — 1 
43.a)0 b) 1680 c)1 d) 1024 45.34 


Section 2.5 


1. a) Countably infinite, —1, —2, —3, —4,... b) Countably 
infinite, 0, 2, —2, 4, —4, ... c) Countably infinite, 
99, 98, 97, ... d) Uncountable_ e) Finite f) Countably infi- 
nite, 0,7,—7,14,—14,... 3. a) Countable: match n with 
the string of n 1s. b) Countable. To find a correspondence, 
follow the path in Example 4, but omit fractions in the top 
three rows (as well as continuing to omit fractions not in low- 
est terms). c) Uncountable d) Uncountable 5. Suppose m 
new guests arrive at the fully occupied hotel. Move the guest 
in Room n to Room m +n forn = 1, 2,3,...; then the new 
guests canoccupyrooms|1tom. 7. Forn = 1,2,3,..., put 


the guest currently in Room 2n into Room n, and the guest 
currently in Room 2n — | into Room n of the new build- 
ing. 9%. Move the guess currently Room i to Room 2i + 1 
fori = 1, 2,3,.... Put the jth guest from the kth bus into 
Room 2* (Q7 +1). 11. a)A = [1, 2] (closed interval of 
real numbers from 1 to 2), B = [3,4] b) A = [1, 2] U Zt, 
B=[3,4]UZ* c) A=[1, 3], B=[2,4] 13. Suppose that 
A is countable. Then either A has cardinality n for some non- 
negative integer n, in which case there is a one-to-one function 
from A to a subset of Z* (the range is the first n positive in- 
tegers), or there exists a one-to-one correspondence f from 
A to Zt; in either case we have satisfied Definition 2. Con- 
versely, suppose that |A| < |Z*|. By definition, this means 
that there is a one-to-one function from A to Zt, so A has the 
same cardinality as a subset of Z* (namely the range of that 
function). By Exercise 16 we conclude that A is countable. 
15. Assume that B is countable. Then the elements of B can 
be listed as b;, bo, b3,.... Because A is a subset of B, taking 
the subsequence of {b,,} that contains the terms that are in A 
gives a listing of the elements of A. Because A is uncount- 
able, thisisimpossible. 17. Assume that A— B is countable. 
Then, because A = (A — B) U (AN B), the elements of A 
can be listed in a sequence by alternating elements of A — B 
and elements of AM B. This contradicts the uncountability of 
A. 19. We are given bijections f from A to B and g from 
C to D. Then the function from A x C to B x D that sends 
(a, c) to (f(a), g(c)) is a bijection. 21. By the definition 
of |A| < | B|, there is a one-to-one function f : A > B. Simi- 
larly, there is a one-to-one function g : B > C. By Exercise 33 
in Section 2.3, the composition go f : A — C is one-to-one. 
Therefore by definition |A| < |C|. 23. Using the Axiom of 
Choice from set theory, choose distinct elements a1, a2, a3, 
...of A one at a time (this is possible because A is infinite). The 
resulting set {a , a2, a3, .. .} is the desired infinite subset of A. 
25. The set of finite strings of characters over a finite alphabet 
is countably infinite, because we can list these strings in al- 
phabetical order by length. Therefore the infinite set S can be 
identified with an infinite subset of this countable set, which 
by Exercise 16 is also countably infinite. 27. Suppose that 
Aj, A2, A3,... are countable sets. Because A; is countable, 
we can list its elements in a sequence as dj, Gj2, aj3, .... The 
elements of the set )_, Ai can be listed by listing all terms 
aj withi + j = 2, then all terms a;; with i + j = 3, then 
all terms aj; withi + j = 4, and soon. 29. There are a 
finite number of bit strings of length m, namely, 2”. The set 
of all bit, strings is the union of the sets of bit strings of length 
m form = O, 1, 2,.... Because the union of a countable 
number of countable sets is countable (see Exercise 27), there 
are a countable number of bit strings. 31. It is clear from 
the formula that the range of values the function takes on fora 
fixed value of m + n,saym +n = x,is(x —2)(x —1)/24+1 
through (x — 2)(x — 1)/2 + (x — 1), because m can assume 
the values 1, 2,3, ..., (x — 1) under these conditions, and 
the first term in the formula is a fixed positive integer when 
m + n is fixed. To show that this function is one-to-one and 
onto, we merely need to show that the range of values for 
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x + 1 picks up precisely where the range of values for x 
left off, i.c., that f(x — 1,1) +1 = fC, x). We have 
f-1,)4+1= eee 1) +(x-1)+1 = et Se = 
= +1 = fC,x). 33. By the Schréder-Bernstein theo- 
rem, it suffices to find one-to-one functions f : (0, 1) > [0, 1] 
and g: [0, 1] > (0, 1). Let f(x) = x and g(x) = (x + 1)/3. 
35. Each element A of the power set of the set of positive 
integers (i.c., A C Z*) can be represented uniquely by the 
bit string aja2a3..., wherea; = lifi € A anda = 0 
if i ¢ A. Assume there were a one-to-one correspondence 
f:Z* > P(Z*). Forma new bit string s = 15253 .. . by set- 
ting s; to be | minus the ith bit of f (7). Then because s differs 
inthe bit from f (i), 5 is notin the range of f,acontradiction. 
37. For any finite alphabet there are a finite number of strings 
of length n, whenever n is a positive integer. It follows by the 
result of Exercise 27 that there are only a countable number 
of strings from any given finite alphabet. Because the set of 
all computer programs in a particular language is a subset of 
the set of all strings of a finite alphabet, which is a countable 
set by the result from Exercise 16, it is itself a countable set. 
39. Exercise 37 shows that there are only a countable number 
of computer programs. Consequently, there are only a count- 
able number of computable functions. Because, as Exercise 
38 shows, there are an uncountable number of functions, not 
all functions are computable. 


Section 2.6 
1 
1.a)3 x4 b) | 4 [2 0 4 6] d) 1 
3 
oli 21 3a[; 1) bl2 2 -3 
101 18 1 0 2 
1 4 3 9 -4 4 
3.6 7 


Of_4q4 15 -4 1 5. | 9/5 rl 

—3 10 2 —3 -1/5 4/5 

0 2 —-8 6 

1 -8 18 -13 
7.0+A =[0+4;;]=[aij +0] =0+A 9.A+(B+C)= 
[aij + iy +eyj)| = [Gy +b) + cj] = (A+B) + C 
11.The number of rows of A equals the number of 
columns of B, and the number of columns of A 
equals the number of rows of B. 13. A(BC) = 


Yytig (C,Parcr)] = [Ly KeaigParcn] = 
Yo, DyAigharcrt | = be (Xoy4iabar) cr] = (AB)C 
15.4" = |) fl 17.a)Let A = [ajj] and 
B = [b,j]. Then +B = [aj + bij]. We have 
(A + B) = [aj + bj] = [aj] + [bj] = A’ + B’. 
b) Using the same notation as in part (a), we have B'A' = 
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b= byitja| = bx ajgbyi| = (AB)‘, because the 
(i, j)th entry is the (j, i)th entry of AB. 19. The result fol- 
bi|d —b ad —bc 0 

‘| Be 7+ 0) eral 
Co ee elle fl 2. AMA = 
A(A---(A(AA7!)A7!).-- ATA! by the associative law. 
Because AA~! = I, working from the inside shows that 
A”(A7!)") = I. Similarly (A7')"A”_ = _ I. Therefore 
(A")-! = (A7!)", 23. The (i, j)th entry of A + A! 
is ajj + ajj, which equals aj; + aj;j, the (j, i)th entry of 
A +A‘, so by definition A + A’ is symmetric. 25. x; = 1, 
x2 = -1,x3 = -2 


a 
lows because 


27.a)[, 1 1 b)fo 0 1 Of, 11 
 t 4 i oo i 72 
101 00 1 i @ 2 

2%a)li 0 0] D1 o o] Of1 0 O 
1 i 0 101 tt 4 
a a 1 1 0 a 


31.a) AVB=[ajj V bij] = [bij Vaij]=BVA b)AAB= 
[aij A bij) = [bij Aaij]=BAA 33.a) AV (B A C) = 
Laij] V [bij Acij] = laiy V iy A cij)] = Kay V bij) A 
(aij V cig) = [aij V bij) A Lai V ij] = (AV B) A (AVC) 
b) AA (BV ©) = [aj] A [bij V cig] = Laij A ij V Cij)] 
[aij A bij) V (aij A cj) = [ay A bil V [aij 
cj] = (A A BVAA C) 35.A © BO C) 

[Vasia\(Ve(Bar A ert))] = [Vg Ve (dig A bar Aer) 


lv, Ve (dig A bar rcrt)| = lV (V, (aig A by)) A Cri 
(AO B)OC 


to tl > Il 


Supplementary Exercises 


l.a)A b)ANB c)A—B AANB APB 3.Yes 
5. A—(A—B) = A—(AN B) = AN (ANB) =A N (AU 
B) =(AN A)U(ANB)=GU(ANB)=ANB 7.Let 
A {1}, B 0,C {1}. Then (A — B) — C = J, but 
A —(B-—C) = {1}. 9.No. For example, let A = B = 
{a,b}, C = %, and D = {a}. Then (A — B)—(C— D) = 
4 —% =9, but (A — C) — (B — D) = {a, b} — {b} = {a}. 
ll. a) |%| < |ANM B| < |A| < [AU Bl < |U| bd) < 
|A — B| < |A@® B| <|AUB|<|A|+|B| 13. a) Yes, no 
b) Yes,no c) f has inverse with f(a) =3; f(b) = 4, 
f-'(c) = 2, f-'(d) =1; ghasnoinverse. 15. If f is one- 
to-one, then f provides a bijection between S and f(S), so 
they have the same cardinality. If f is not one-to-one, then 
there exist elements x and y in S such that f(x) = f(y). 
Let S = {x, y}. Then |S| = 2 but | f(S)| = 1. 17. Let 
x € A. Then Sp({x}) = (f(y) | y € {x} = (£9). By 


the same reasoning, S,({x}) = {g(x)}. Because Sy = Sy, 
we can conclude that {f(x)} = {g(x)}, and so necessarily 
f(x) = g(x). 19. The equation is true if and only if the 
sum of the fractional parts of x and yislessthan1. 21. The 
equation is true if and only if either both x and y are in- 
tegers, or x is not an integer but the sum of the fractional 
parts of x and y is less than or equal to 1. 23. If x is an 
integer, then [x] + |[m —x] = x +m-—x = m. Oth- 
erwise, write x in terms of its integer and fractional parts: 
x =n-+e, wheren = |x| andO <e < 1. Inthis case |x| + 
[m—x]=|n+e]+|m—-—n-—e|]=n+m—-—n-1=m-1. 
25. Write n = 2k + 1 for some integer k. Then n? = 4k? + 
4k+1,son?/4 =k? +k+ 4. Therefore, [n?/4] =k? +k+1. 
But (n?+3)/4 = (4k7+4k+143)/4=k?+k+1. 27. Let 
x =n-+ (r/m) + €, where n is an integer, r is a nonnegative 
integer less than m, and € is areal number with O < € < 1/m. 
The left-hand side is [nm +r + me] =nm-+r. On the right- 
hand side, the terms |x| through |x + (m+r—1)/m| are all 
just n and the terms from |x + (m —r)/m| on are alln + 1. 
Therefore, the right-hand side is (m—r)n+r(n+1) =nm-+r, 
as well. 29.101 3l.a; = 1; dont1 = n- a2» for all 
n > OQ; and doa, = n + Go,_; for alln > O. The next 
four terms are 5346, 5353, 37471, and 37479. 33. If each 
f—'(J) is countable, then S = f7'(1) U f7!Q)U--- is 
the countable union of countable sets and is therefore count- 
able by Exercise 27 in Section 2.5. 35. Because there is 
a one-to-one correspondence between R and the open inter- 
val (0, 1) (given by f(x) = 2 arctan(x)/z), it suffices to 
shows that |(0, 1) x (0, 1)| = |(0, 1)|. By the Schréder- 
Bernstein theorem it suffices to find injective functions f : 
(0,1) ~ (0, 1) x (0, 1) and g : (0,1) x (0,1) > (0, 1). 
Let f(x) = (, 3): For g we follow the hint. Suppose 
(x, y) € (0, 1) x (O, 1), and represent x and y with their deci- 
mal expansions x = 0.x,x2x3...and y = 0.y; y2y3..., never 
choosing the expansion of any number that ends in an infinite 
string of 9s. Let g(x, y) be the decimal expansion obtained by 
interweaving these two strings, namely 0.x) yj.x2y2x3y3.... 


37. At” = k | ; Atetl = E i , Aant2 


0 1 -1 0 


0 0 
that A = k i] . Let B = k | . Because AB = BA, 


& 4 , AM — : ry ,forn >0 39. Suppose 


c 0 O 


0 0 


it follows that c = 0 anda = d. Let B = I‘ 0 


] . Because 


AB = BA, it follows that b = 0. Hence, A = E ‘| = al. 


41.a)Let A OO = [bij]. Then bij = (aj, A 0) 
V-++V (dip AO) = 0. Hence, AO 0 = 0. Similarly OO A = 0. 
b)Av 0 = [aij V0] = [aij] = A. Hence AV 0 = A. 
SimilarlyOv A =A. c) AAO = [ajj A 0] = [0] = 0. Hence 
AA0=0. Similarly 0A, A = 0. 


CHAPTER 3 


Section 3.1 


1. max := 1,7 := 2, max := 8,i := 3, max := 12,7 := 4, 
$2= 5,0 2= 6,7 2= 7, max] 14,1 2= 83.4 2=— 9.7 t= 10; 
fa ml 
3. procedure AddUp(aj, .. 
sum := a, 
fori :=2ton 
sum := sum + aj 
return sum 


. , Gy: integers) 


5. procedure duplicates(a,a2,.. 
nondecreasing order) 
k := 0 {this counts the duplicates} 
ji=2 
while j <n 
if aj; = aj-1 then 
k:=k+1 
Ck = Qj 
while j <n and aj = cx 
ji=jt+l 
ji=jt+l 
{c1, C2, ..., Cg 18 the desired list} 
7. procedure /ast even location(a,,a2, .. 
k:=0 
for i := 1 ton 
if a; is even then k :=i 
return & {k = 0 if there are no evens} 
9. procedure palindrome check(a,d2 .. . an: string) 
answer := true 
for i := 1 to [n/2] 
if a; A d,+\~; then answer := false 
return answer 
11. procedure interchange(x, y: real numbers) 


., dy: integers in 


. ,dn: integers) 


i= % 
xy 
Yiez 
The minimum number of assignments needed is three. 
13. Linear search: i := 1,i := 2,i := 3,i := 4,7 := 5, 
i := 6,7 := 7, location := 7; binary search: i := 1, j := 8, 
m:=4,i:=5,m:=6,i:=7,m:=7, j :=7, location := 7 


15. procedure insert(x, a1, a2, .. 
{the list is in order: aj < a2 <--- 
Anti = xX+ 1 
it 1 
while x > a; 

i:=i+l1 
for j:=Oton—i 


., Qn: integers) 
< dy} 


Gn—j+l += 4n-j 
aj =x 
{x has been inserted into correct position} 


17. 


19. 


21. 


23. 


25. 


27. 
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procedure first largest(a,, ..., dy: integers) 
max := a, 
location := 1 
for i :=2ton 
if max < a; then 
max := Qj 
location := i 
return location 


procedure mean-median-max-min(a, b, c: integers) 
mean := (a+b+c)/3 
{the six different orderings of a, b, c with respect 
to > will be handled separately} 
if a > b then 
if b > c then median : 


b; max := a; min := c 


(The rest of the algorithm is similar.) 


procedure first-three(a,, a2, ..., dy: integers) 
if aj > az then interchange a, and ay 
if a2 > a3 then interchange a2 and a3 
if aj > az then interchange a, and a2 


procedure onto( f: function from A to B where 
Av= {Gis 3043; Gn)s Bo= {D100 Dints Gs ote y Gis 
b,,..., bm are integers) 
fori :=1tom 
hit(b;) :=0 
count := 0) 


for j :=1ton 
if hit( f (a;)) = 0 then 
hit(f (aj)) := 1 
count := count + | 
if count = m then return true else return false 


procedure ones(a: bit string, a = aja2...dn) 
count:= 0 
fori := 1 ton 
if a; := | then 
count := count + | 
return count 
procedure ternary search(s: integer, a,,d2,..., Gn: 
increasing integers) 
ial | 
j2=n 
while i < j —1 
1:= [G+ j)/3] 
u:= [2G + j)/3] 
if x >a, theni :=u+1 
else if x > a; then 


i:=/+1 
ji=u 
else j :=/ 


if x = a; then location := i 
else if x = a; then location := j 
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else location := 0 
return Jocation {0 if not found} 


29. procedure find a mode(a,, a2, ... , An: nondecreasing 
integers) 
modecount := 0 
je | 
while i <7 
value := aj 
count := 1 
while i < n anda; = value 
count := count + 1 
i:=it+1 
if count > modecount then 
modecount := count 
mode := value 
return mode 
31. procedure find duplicate(a,, az, ..., An: integers) 
location := 0 
i:=2 
while i < n and location = 0 
j=l 
while j <i and location = 0 
if a; = a; then location := i 
else j:=j+1 
i:=it+l1 
return /ocation 
{location is the subscript of the first value that 
repeats a previous value in the sequence} 
33. procedure find decrease(a\, a2, ..., An: positive 
integers) 
location := 0 
122 
while i < n and location = 0 
if aj < aj—, then location := i 
else i :=i+1 
return /ocation 
{location is the subscript of the first value less than 
the immediately preceding one} 
35. At the end of the first pass: 1, 3, 5, 4, 7; at the end of the 
second pass: 1, 3, 4, 5, 7; at the end of the third pass: 1, 3, 4, 
5, 7; at the end of the fourth pass: 1, 3, 4, 5, 7 
37. procedure better bubblesort(a,, ..., dy: integers) 
i:= 1; done: = false 
while i < n and done = false 
done : = true 
forj:=1ton—i 
if aj; > aj+i then 
interchange a; and aj+ 
done : = false 
i:=it+l 
{a,,..., Gp 1S in increasing order} 


> 


5; 7,4 
at the end of the fourth pass: 1,3,4,5,7 41. a) 1,5, 4, 3, 
2) 1,,..2,.4,.3,5% 1, 25.3,-4,- 95 1,23. 324.5: - bb) 1,4 3, 2. 
33 dg 2s 3: od, 2,354, Se 1,2; 
5 2.3 : 2, 


; 4,5; 1 
31, 2, 3, 4, 5; 1, 2, 3, 4,55 1 


out the linear search algorithm given as Algorithm 2 in this 
section, except that we replace x 4 aj by x < aj, and 
we replace the else clause with else location := n + 1. 
45.24+3+4+4+4+---+n = (n?+n—2)/2 47. Find the 
location for the 2 in the list 3 (one comparison), and insert it 
in front of the 3, so the list now reads 2, 3, 4, 5, 1, 6. Find 
the location for the 4 (compare it to the 2 and then the 3), 
and insert it, leaving 2, 3, 4, 5, 1, 6. Find the location for the 
5 (compare it to the 3 and then the 4), and insert it, leaving 
2,3, 4,5, 1, 6. Find the location for the 1 (compare it to the 
3 and then the 2 and then the 2 again), and insert it, leaving 
1, 2, 3, 4, 5, 6. Find the location for the 6 (compare it to the 
3 and then the 4 and then the 5), and insert it, giving the final 
answer |, 2, 3,4, 5, 6. 
49. procedure binary insertion sort(a,, a2, ..., An: 
real numbers with n > 2) 
for j :=2ton 
{binary search for insertion location 7} 
left := 1 
right := j — 1 
while left < right 
middle := | (left + right) /2| 
if aj > dmiddte then left := middle + 1 
else right := middle 
if aj < diez then i := left else i := left+ 1 
{insert a; in location i by moving a; through a;_—1 
toward back of list} 
m:= aj 


fork :=Otoj—i-1 
Aj—k ‘= Aj—k-1 

qj:=m 

{a,, d2,..., Ay are sorted} 


51. The variation from Exercise 50 53. a) Two quarters, one 
penny b) Two quarters, one dime, one nickel, four pennies 
c) A three quarters, one penny d) Two quarters, one dime 
55. Greedy algorithm uses fewest coins in parts (a), (c), and 
(d). a) Two quarters, one penny b) Two quarters, one dime, 
nine pennies c) Three quarters, one penny d) Two quarters, 
one dime 57. The 9:00-9:45 talk, the 9:50—-10:15 talk, the 
10:15—10:45 talk, the 11:00-11:15 talk 59. a) Order the 
talks by starting time. Number the lecture halls 1, 2, 3, and 
so on. For each talk, assign it to lowest numbered lecture hall 
that is currently available. b) If this algorithm uses n lecture 
halls, then at the point the nth hall was first assigned, it had 
to be used (otherwise a lower-numbered hall would have been 
assigned), which means that talks were going on simulta- 
neously (this talk just assigned and the n — 1 talks currently 
in halls 1 throughn — 1). 61. Here we assume that the men 
are the suitors and the women the suitees. 
procedure stable(M,, M2,..., Ms, Wi, W2,..., Ws: 

preference lists) 
fori := 1 tos 

mark man i as rejected 
for i := 1 tos 

set man 7’s rejection list to be empty 
for j := 1tos 


set woman j’s proposal list to be empty 
while rejected men remain 
for i := 1 tos 
if man i is marked rejected then add i to the 
proposal list for the woman j who ranks highest 
on his preference list but does not appear on his 
rejection list, and mark i as not rejected 
for j :=1tos 
if woman j’s proposal list is nonempty then 
remove from j’s proposal list all men i 
except the man ig who ranks highest on her 
preference list, and for each such man i mark 
him as rejected and add j to his rejection list 
for j := 1 tos 
match j with the one man on j’s proposal list 
{This matching is stable.} 
63. If the assignment is not stable, then there is aman m anda 
woman w such that m prefers w to the woman w’ with whom 
he is matched, and w prefers m to the man with whom she is 
matched. But m must have proposed to w before he proposed 
to w’, because he prefers the former. Because m did not end 
up matched with w, she must have rejected him. Women re- 
ject a suitor only when they get a better proposal, and they 
eventually get matched with a pending suitor, so the woman 
with whom w is matched must be better in her eyes than m, 
contradicting our original assumption. Therefore the marriage 
is stable. 65. Run the two programs on their inputs concur- 
rently and report which one halts. 


Section 3.2 


1. The choices of C and k are not unique. a) C = 1,k = 10 
b) C=4,k =7c) Nod) C=5,k=le)C=1,k=0f)C 

Lk=2 3.x4+9x344x+7 < 4x4 forall x > 9; witnesses 
C=4,k=9 5. (x? +1/@41) =x—-14-2/(¢4+) <x 
for all x > 1; witnesses C = 1,k =1 7. The choices of C 
and k are not unique. a)n 3,.C 3,k 1 b)n 3, 
C=4,k=1lon=1,C=2,k=1d)n=0,C=2,k=1 
9.x? + 4x +17 < 3x3 forall x > 17, s0 x? + 4x + 17 is 
O(x3), with witnesses C = 3, k = 17. However, if x? were 
O(x? + 4x + 17), then x? < C(x? + 4x +17) < 3Cx? for 
some C, for all sufficiently large x, which implies that x < 3C 
for all sufficiently large x, which is impossible. Hence, x? is 
not O(x7+4x +17). 11.3x4+4+1 < 4x4 = 8(x+/2) for all 
x > 1,80 3x4 + 1 is O(x*/2), with witnesses C = 8, k = 1. 
Also x*/2 < 3x4+1 forall x > 0,so.x*/2 is O(3x++1), with 
witnessesC = 1,k=0. 13. Because 2” < 3” foralln > 0, 
it follows that 2” is O(3”"), with witnesses C = 1,k = 0. 
However, if 3” were O(2”), then for some C, 3” < C - 2” for 
all sufficiently large n. This says that C > (3/2)” for all suffi- 
ciently large n, which is impossible. Hence, 3” is not O(2”). 
15. All functions for which there exist real numbers k and C 
with | f(x)| < C for x > k. These are the functions f (x) that 
are bounded for all sufficiently large x. 17. There are con- 
stants Cy, C2, ky, and kz such that | f(x)| < C1|g(x)| for all 
x > k, and |g(x)| < C2|h(x)| for all x > ko. Hence, for x > 
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max(k,, k) it follows that | f(x)| < Ci|g(x)| < C1 C2|hA(x)|. 
This shows that f(x) is O(h(x)). 19. 2"+! is O(2"); 
27" isnot. 21. 1000logn, Jn, n logn, n*/1000000, 2”, 
3”, 2n! 23. The algorithm that uses n log n operations 
25.a) O(n?) b) O(n?) ©) O(n? -n!) 27.a) O(n? logn) 
b) O(n2(log n)?) ¢) O(n2") 29. a) Neither ©(x?) nor 
Q(x?) b) O(x?) and Q(x?) ¢) Neither Q(x?) nor Q(x?) 
d) Q(x”), but not O(x2)  e) Q(x”), but not O(x?) f) Q(x?) 
and @(x?) 31. If f(x) is O(g(x)), then there exist con- 
stants Cy; and C2 with Ci|g(x)| < |f()| < Colg(x)|. 
It follows that | f(x)| < Calg(x)| and |g(x)| < (1/C1)I f 0 
forx > k. Thus, f(x) is O(g(x)) and g(x) is O(f (x)). Con- 
versely, suppose that f(x) is O(g(x)) and g(x) is O(f(x)). 
Then there are constants C), C2, ki, and k2 such that | f (x)| < 
Ci|g(x)| forx > k; and|g(x)| < Co| f(x)| forx > ko. Wecan 
assume that C2 > 0 (we can always make C> larger). Then we 
have (1/C2)lg(x)| < If (x) < Cilg(x)| for x > max(ky, ky). 
Hence, f(x) is O(g(x)). 33. If f(x) is O(g(x)), then f(x) 
is both O(g(x)) and Q(g(x)). Hence, there are positive con- 
stants Cy, kj, C2, and kz such that | f(x)| < C2|g(x)| for 
allx > kz and |f(x)| > Cil|g(x)| for all x > ky. It fol- 
lows that C;|g(x)| < |f(x)| < C2|g(x)| whenever x > k, 
where k = max(k;,k2). Conversely, if there are positive con- 
stants Cj, C2, and k such that C;|g(x)| < | f(x)| < Calg(x)| 
for x > k, then taking kj = kz = k shows that f(x) is both 
O(g(x)) and O(g(x)). 


35. » 


k x 


37. If f(x) is O(1), then | f(x)| is bounded between pos- 
itive constants C; and C2. In other words, f(x) cannot 
grow larger than a fixed bound or smaller than the nega- 
tive of this bound and must not get closer to 0 than some 
fixed bound. 39. Because f(x) is O(g(x)), there are con- 
stants C and k such that |f(x)| < Clg(x)| forx > k. 


Hence, |f"(x)| < C"|g"(x)| forx > k, so f"(x) is 
O(g"(x)) by taking the constant to be C”. 41. Because 
f (x) and g(x) are increasing and unbounded, we can assume 
f(x) = land g(x) = 1 for sufficiently large x. There are 
constants C and k with f(x) < Cg(x) forx > k. This 
implies that log f(x) < logC + log g(x) < 2 log g(x) 
for sufficiently large x. Hence, log f(x) is O(log g(x)). 
43. By definition there are positive constraints Cy, Cis 
C2, C5, ki, ki, ko, and k, such that fi(x) > Cilg(x)| 
for all x > ky, fi(x) < Cilg(x)| for all x > ky, 
fxlx) = Colg(x)| for all x > ky, and fr(x) < Chlg(x)| 
for all x > k’. Adding the first and third inequalities shows 
that fi(x) + fo(x) => (Cy + C2)|g(x)| for all x > k where 
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k = max(ky, kz). Adding the second and fourth inequalities 
shows that fi (x) + fo(x) < (C) + C5)|g(x)| for all x > k’ 
where k’ = max(k}, k5). Hence, fi(x) + f2(x) is O(g(x)). 
This is no longer true if f; and f can assume negative values. 
45. This is false. Let f) = x? + 2x, fox) = x7 4+ x, 
and g(x) = x*. Then fj(x) and f(x) are both O(g(x)), 
but (fi — f2)(x) is not. 47. Take f(n) to be the func- 
tion with f(n) = n if n is an odd positive integer and 
f(n) = 1 ifn is an even positive integer and g(n) to be the 
function with g(n) = 1 if n is an odd positive integer and 
g(n) = n ifn is an even positive integer. 49. There are 
positive constants C1, C2, C}, Ch, ki, ky, k2, and ki, such that 
IfiQ®)| = Cilgi@)| for all x > ky, fiG)| < Cilgi@)| 
for all x > Ki, | fo(x)| > C2\go(x)| for all x > ko, and 


| fo(x)| < CSlgo(x)| for all x > k5. Because fy and go 
are never zero, the last two inequalities can be rewritten as 
|1/fo(x)| < (1/C2)|1/g2(«)] for all x > ky and |1/fo(x)| > 
(1/C5)|1/g2(x)| for all x > k,. Multiplying the first and 
rewritten fourth inequalities shows that | fi(x)/fo(x)| = 
(C1 /C})|g1(x)/g2(x)| for all x > max(k;,k4), and mul- 
tiplying the second and rewritten third inequalities 
gives | fi(x)/fala)| < (Ci/C2)|gi(@)/g2(x)| for all x > 
max(k), k2). It follows that f|/f2 is big-Theta of g/g. 
51. There exist positive constants C1, C2, ki, ko, ky, ki 
such that | f(x, y)| < Cilg(x, y)| for all x > k; and y > ko 
and | f(x, y)| = Calg(x, y)| for all x > k, and y > kj. 
53. (x7 + xy + x logy)? < (3x?y3) = 27x%y? for 
x > land y> 1, because x2 < x?y,xy < xy, and 
xlogy <x?y. Hence, (x7 +xy+xlogy)? is O(x®y3). 
55. For all positive real numbers x and y, [xy] < xy. 
Hence, [xy] is O(xy) from the definition, taking C = 1 
and kj =k, =0. 57. Clearly n? < n° for alln > 2; 
therefore n4 is O(n‘). The ratio n?/n©° = n° is un- 
bounded so there is no constant C such that n@ < Cn¢ for 
largen. 59. If f and g are positive-valued functions such 
that limy-+o00 f(x)/g(x) = C < oo, then f(x) < (C+1)g(x) 
for large enough x, so f(n) is O(g(n)). If that limit is 00, 
then clearly f(n) is not O(g(n)). Here repeated applica- 
tions of L’H6pital’s rule shows that lim, xf /b* = 0 
and limy soo b*/x4 = oo. G6l.a)limy so x7/x3 = 


limyso0 1/x = 0 Db) limy—.o0 ap = lim, “2% = 


limy-so0 oe = 0 (using L’H6pital’s rule) c) lim, = = 


limy+00 nT = lims+oo FAH? = 0 (using L'Hopital’s 


rule) d) limyoo =A¥#! = limy 500 (1 +14 +) Se) 


63. » 


65. No. Take f(x) = 1/x? and g(x) = 1/x. 67. a) Be- 
cause limy.o. f(x)/g(x) = 0, If()I/lg@)| < 1 for 
sufficiently large x. Hence, |f(x)| < |g(x)| forx > k 
for some constant k. Therefore, f(x) is O(g(x)). b) Let 
f(x) = g(x) = x. Then f(x) is O(g(x)), but f(x) is 
not o(g(x)) because f(x)/g(x) = 1. 69. Because f2(x) is 
o(g(x)), from Exercise 67(a) it follows that f2(x) is O(g(x)). 
By Corollary 1, we have f(x) + fo(x) is O(g(x)). 71. We 
can easily show that (n —7)(i+ 1) >n fori =0,1,...,2—1. 
Hence, (n!)? = (n- 1)((n — 1) -2)- (n — 2)-3)-:-2-(n— 
1))-(_-n) =n". Therefore, 2logn! > nlogn. 73. Compute 
that log 5! * 6.9 and (5 log 5)/4 ~% 2.9, so the in- 
equality holds forn = 5. Assume n > 6. Because n! 
is the product of all the integers from n down to 1, we 
have n! > n(n — 1)(n — 2)--- [n/2] (because at least 
the term 2 is missing). Note that there are more than n/2 
terms in this product, and each term is at least as big as 
n/2. Therefore the product is greater than (n/2)"/?). Tak- 
ing the log of both sides of the inequality, we have logn! > 
log?) = Slog 5 = $(logn — 1) > (nlogn)/4, because 
n > 4implies logn — 1 > (logn)/2. 75. All are not 
asymptotic. 


Section 3.3 


1.00) 3.07) 5.2n—-1 7.Linear 9. O(n) 
11. a) procedure disjointpair(S,, S2,..., Sn: 
subsets of {1,2,...,}) 
answer := false 
for i := 1 ton 
for j:=i+1ton 
disjoint := true 
for k := 1 ton 
ifk ¢ S; andk ¢€ S; then disjoint := false 
if disjoint then answer := true 
return answer 


b) O(n?) 13.a)power := l,y := Li := 1, 
power : 2, y:=3;i:=2, power := 4,y := 15 
b) 2n multiplications andn additions 15. a) 2! ~103* 10° 
b) 10° =e) 3.96 x 107. d)3.16 x 10+ e)29 f)12 
17. a) 260! b) 760-10!" ©) [2¥60-10° | ~2x 102331768 
d) 60,000,000 e) 7,745,966 f)45 g)6 19. a) 36 years 
b) 13 days c)19 minutes 21. a)Less than 1 millisec- 
ond more b) 100 milliseconds more c) 2n + 1 milliseconds 
more d) 3n2+3n-+ 1 milliseconds more e) Twice as much 
time f) 22”"*+! times as many milliseconds g)n + 1 times 
as many milliseconds 23. The average number of compar- 
isons is (3n+4)/2. 25. O(logn) 27. O(n) 29. O(n?) 
31. O(n) 33. O(n) 35. O(log n) comparisons; O(n?) 
swaps 37. O(n?2") 39.a) doubles _ b) increases by 1 
41. Use Algorithm 1, where A and B are now n x n up- 
per triangular matrices, by replacing m by n in line 1, and 


having q iterate only from i to j, rather than from | to k. 
43. nn+1I(n+2)/6 45. A((BC)D) 


Supplementary Exercises 


1. a) procedure last max(a\, ..., Gn: integers) 
max := a, 
last := 1 
i:=2 
while i <n 
if a; > max then 
max := Qj 
last :=1 
i:=i+l 
return /ast 


b) 2n — 1 = O(n) comparisons 


3. a) procedure pair zeros(b,b2 ... by: bit string, n > 2) 


x= by 

y= bo 

k:=2 

while k <n and (x 4 0or y £0) 
k:=k+1 

xi=y 

y = dk 


if x = O and y = 0 then print “YES” 
else print “NO” 


b) O(n) 


5. a) and b) 
procedure smallest and largest(a1, a2, ..., An: integers) 
min := ay 
max := a, 
for i :=2ton 
if a; < min then min := a; 
if a; > max then max := aj 
{min is the smallest integer among the input, and max is the 
largest} 


c)2n—2 


7. Before any comparisons are done, there is a possibility 
that each element could be the maximum and a possibility 
that it could be the minimum. This means that there are 2n 
different possibilities, and 2n — 2 of them have to be elimi- 
nated through comparisons of elements, because we need to 
find the unique maximum and the unique minimum. We clas- 
sify comparisons of two elements as “virgin” or “nonvirgin,” 
depending on whether or not both elements being compared 
have been in any previous comparison. A virgin comparison 
eliminates the possibility that the larger one is the minimum 
and that the smaller one is the maximum; thus each virgin 
comparison eliminates two possibilities, but it clearly cannot 
do more. A nonvirgin comparison must be between two ele- 
ments that are still in the running to be the maximum or two 
elements that are still in the running to be the minimum, and 
at least one of these elements must not be in the running for 
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the other category. For example, we might be comparing x 
and y, where all we know is that x has been eliminated as 
the minimum. If we find that x > y in this case, then only 
one possibility has been ruled out—we now know that y is 
not the maximum. Thus in the worst case, a nonvirgin com- 
parison eliminates only one possibility. (The cases of other 
nonvirgin comparisons are similar.) Now there are at most 
|n/2] comparisons of elements that have not been compared 
before, each removing two possibilities; they remove 2|n/2| 
possibilities altogether. Therefore we need 2n — 2 — 2|n/2| 
more comparisons that, as we have argued, can remove only 
one possibility each, in order to find the answers in the worst 
case, because 2n — 2 possibilities have to be eliminated. This 
gives us a total of 2n — 2 — 2|n/2]| + |[n/2] comparisons in 
all. But 2n — 2 —2|n/2] + |[n/2] = 2n —2— |n/2| =2n—- 
2+ [—n/2] =[2n — n/2] — 2 = [3n/2] — 2, as desired. 
9. The following algorithm has worst-case complexity O(n*). 
procedure equal sums(a\, a2,..., dn) 
fori :=1ton 
for j :=i+ 1 ton {since we wanti < j} 
for k :=1ton 
for /:=k+ 1 ton {since we want k </} 
if aj + aj = ax + q and (i, j) ¥ (k,!) 
then output these pairs 
11. At end of first pass: 3, 1, 4, 5, 2, 6; at end of second 
pass: 1, 3, 2, 4, 5, 6; at end of third pass: 1, 2, 3, 4, 5, 6; 
fourth pass finds nothing to exchange and algorithm termi- 
nates 13. There are possibly as many as n passes through 
the list, and each pass uses O(n) comparisons. Thus there 
are O(n?) comparisons in all. 15. Because logn < n, we 
have (n logn + nyo < (n? +n?) < (2n?)3 = 8n° for 
alln > 0. This proves that (n logn + n*)> is O(n®), with 
witnesses C = 8andk = 0. 17. O(x?2*) 19. Note that 
Pea kM BFP Se Le. 1-5=4. 21. Allofthese 
functions are of the same order. 23.2!°7 25. (log ny’, 
2V 1082" n(log n)!00!, 71-0001 1 0001", n" 27. For exam- 
ple, f(n) — n2ln/2\+1 and g(n) _ n2in/2] 


29. a) 
procedure brute(a,, a2, ..., dy : integers) 
fori :=1ton—1 
for j:=i+1ton 


for k:= 1ton 

if a; + a; = a, then return true else return false 
b) O(n?) 
31. For m,: w and w2; for m2: w; and w3; for m3: w2 and 
w3; for wy: my and mo; for w2: m, and m3; for w3: m2 and 
m3 33. A matching in which each woman is assigned her 
valid partner ranking highest on her preference list is female 
optimal; a matching in which each man is assigned his valid 
partner ranking lowest on his preference list is male pessi- 
mal. 35. a) Modify the preamble to Exercise 60 in Sec- 
tion 3.1 so that there are s men m ,m2,..., ms and t women 
W1,W2,..., w;. A matching will contain min(s, t) marriages. 
The definition of “stable marriage” is the same, with the un- 
derstanding that each person prefers any mate to being un- 
matched. b) Create |s — ¢| fictitious people (men or women, 
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whichever is in shorter supply) so that the number of men 
and the number of women become the same, and put these 
fictitious people at the bottom of everyone’s preference lists. 
c) This follows immediately from Exercise 63 in Section 3.1. 
37.5;15 39. The first situation in Exercise 37 41. a) For 
each subset S of {1, 2,...,}, compute Vies wj;. Keep track 
of the subset giving the largest such sum that is less than or 
equal to W, and return that subset as the output of the algo- 
rithm. b) The food pack and the portable stove 43. a) The 
makespan is always at least as large as the load on the proces- 
sor assigned to do the lengthiest job, which must be at least 
max j=1,2,....n tj. Therefore the minimum makespan satisfies 
this inequality. b) The total amount of time the processors 
need to spend working on the jobs (the total load) is aes tj. 
Therefore the average load per processor is s ed tj. The 
maximum load cannot be any smaller than the average, so the 
minimum makespan is always at least this large. 45. Pro- 
cessor |: jobs 1, 4; processor 2: job 2; processor 3: jobs 3, 5 


CHAPTER 4 


Section 4.1 


l.a) Yes b) No c) Yes d)No __ 3. Suppose that a | b. Then 
there exists an integer k such that ka = b. Because a(ck) = bc 
it follows thata | bc. 5.Ifa|bandb | a, there are integers 
c and d such thatb = ac anda = bd. Hence, a = acd. 
Because a ¥ 0 it follows that cd = 1. Thus either c = d = 1 
orc =d=-—1.Hence,eithera =bora=-—b. 7. Because 
ac | bc there is an integer k such that ack = bc. Hence, ak = b, 
soa|b. 9.a)2,5 b)—I11, 10 c) 34, 7 d)77, 0 e)0, 0 
f)0, 3 g)—1, 2 h)4, 0 I1.a)7:00 b) 8:00 c) 10:00 
13.a)10 b)8 c)0 d)9 e)6 f)ll 15.Ifamodm = 
b mod m, then a and b have the same remainder when di- 
vided by m. Hence, a = qym +r and b = qom +1, where 
0 <r <m.Itfollows that a—b = (qi —q2)m,som | (a—b). 
It follows thata = b (mod m). 17. There is some b with 
(b — 1)k <n < bk. Hence, (b — 1)k < n—1 < bk. Divide 
by k to obtain b — 1 < n/k < bandb—1 < (n—1)/k <b. 
Hence, [n/k] = band |(n—1)/k] =b-—1. 19.x modm 
if xmodm < [m/2] and (x mod m) — m if x mod m > 
[m/2] 21. a)1l b)2 c)3 d)9 23. a) 1, 109 b) 40, 
89 c) —31, 222 d) —21,38259 25. a) —15 b) —7 c) 140 
27. —1, —26, —51, —76, 24, 49, 74,99 29. a)No b) No 
c) Yes d)No 31. a) 13 a)6 33. a)9 b)4 c)25 dO 
35. Let m = tn. Because a = b (mod m) there exists an 
integer s such thata = b + sm. Hence,a = b + (st)n, 
soa = b(modn). 37.a)Letm = c = 2,a = 0, 
and b = 1. ThenO = ac = bc = 2 (mod 2), but 
0=a#b=1(mod2). b)Letm=5,a=b=3,c=1, 
and d = 6. Then 3 = 3 (mod 5) and 1 = 6 (mod 5), but 
31 =344=729 = 3° (mod 5). 39. By Exercise 38 the 
sum of two squares must be either0 +0 = 0,0+ 1 = 1, 
or 1+ 1 = 2, modulo 4, never 3, and therefore not of the 
form 4k + 3. 41. Because a = b(modm), there exists an 


integer s such thata = b+ sm,soa — b = sm. Then 
ak — bk = (a — b)(ak“! + a2 +. --- 4 abi? + DE, 
k > 2,is also a multiple of m. It follows that a* = p* (mod m). 
43. To prove closure, note that a -,, b = (a-b) mod m, which 
by definition is an element of Z,,. Multiplication is associa- 
tive because (a +) b) +m c anda -m (b -m c) both equal 
(a - b-c) mod m and multiplication of integers is associa- 
tive. Similarly, multiplication in Z,, is commutative because 
multiplication in Z is commutative, and | is the multiplicative 
identity for Z,,, because 1 is the multiplicative identity for Z. 
45.0+50 = 0,0+51 = 1,0+52 = 2,0+53 = 3,0+54 = 
4; 1451 = 2, 1452 = 3, 1+53 = 4, 1454 = 0; 2+52 = 
4,2+53 = 0, 2+54 = 1; 3453 = 1, 3+54 = 2; 4444 = 3 
and 0-50 = 0, 0-51 = 0, 0-52 = 0, 0-53 = 0, 0-54 = 0; I-51 

1, 1-52 = 2, 1-53 = 3, 1-54 = 4; 2-52 = 4, 2-53 = 1, 2-54 
3; 3-53 = 4,354 = 2;454 = 1 47. f is onto but not 
one-to-one (unless d = 1); g is neither. 


Section 4.2 


J.a) 11100111 b) 10001 10110100 ce) 10111 11010110 
1100 3.a)31 b)513 c)341 d) 26,896 5. a)1 O111 
1010 b)11 1000 0100 c)1 0001 0011 d)101 0000 
1111 7.a) 1000 0000 1110 b)1 0011 0101 1010 1011 
c) 10101011 1011 1010 d) 1101 1110 1111 1010 11001110 
1101 9.1010 1011 1100 1101 1110 1111 = 11. (B7B)46 
13. Adding up to three leading Os if necessary, write the binary 
expansion as (... b3b22b21b20b13b12b11b10b03b02b01bo0)2- 
The value of this numeral is bo9 + 2bo, + 4bo2 + 803 + 
2 big + 2°by, + 2°bi2 + 27b13 + 28bo9 + 22bo1 + 2! bor + 


2bo3 + ++. , which we can rewrite as boo + 
2bo, + 4bo2 + 8bo3 + (bin + 2b1; + 4bi2 + 8543) 
2+ + (bo9 + 2b2) + 4b22 + 823) - 28 + --- . Now 


(bj3bj2bi,bi9)2 translates into the hexadecimal digit h;. 
So our number is fg + hy - 4 ho - B4... = 
ho + hy - 16 + hz - 16? + ---, which is the hex- 
adecimal expansion (... AiAiho)i6. 15 Adding up to 
two leading Os if necessary, write the binary expansion as 
(... b29b21b29b12b11b109b02b01bo9)2. The value of this nu- 
meral is bo +2b91 + 4b02 + 23b19 + 2411 +2°b12 + 2° b29 
2" bo, + 28bo9 +--+, which we can rewrite as boy + 2bo + 
Aboo + (b19 + 2b11 +4012) - 27 + (b29 +2b21 +4b22)-2°+- +>. 
Now (bj2b;,b;0)2 translates into the octal digit h;. So our num- 
ber isho+hy-2? +h2-26+--- = ho thi -8+h2-874+---, 
which is the octal expansion (... hihjho)g. 17.1 1101 
1100 1010 1101 0001, 1273)g 19. Convert the given octal 
numeral to binary, then convert from binary to hexadecimal 
using Example 7. 21. a) 1011 1110, 10 0001 0000 0001 
b) 1 1010 1100, 1011 0000 0111 0011 c) 100 1001 1010, 
101 0010 1001 0110 0000 d) 110 0000 0000, 
10000000 0001 11111111 23. a) 1132, 144,305 b) 6273, 
2,134,272 c)2110, 1,107,667 d) 57,777, 237,326,216 
25.436 27.27 29. The binary expansion of the integer is 
the unique such sum. 31. Leta = (Gn—1dy—2...a41a0)10.- 
Then a = 10°-1ay_1 + 10°~2an_2 + ++» + 10a, + ao 
= dn—-1 + Gn—-2 + -:: + a; + ao (mod 3), because 


10/_ = 1 (mod 3)) for all nonnegative integers j. It fol- 
lows that 3 | a if and only if 3 divides the sum of the dec- 
imal digits of a. 33. Let d = (Gn—1dn—2...a,a9)2. Then 
a =ag + 2a, + 22a. + --- +2" -1a,_-) = ag -—ay t+ am— 
a3 + --- + a,_; (mod 3). It follows that a is divisible by 
3 if and only if the sum of the binary digits in the even- 
numbered positions minus the sum of the binary digits in the 
odd-numbered positions is divisible by 3. 35.a) —6 b) 13 
c)—14 d)0 37. The one’s complement of the sum is found 
by adding the one’s complements of the two integers except 
that a carry in the leading bit is used as a carry to the last bit 
of the sum. 39. If m > 0, then the leading bit a,—1 of the 
one’s complement expansion of m is 0 and the formula reads 
m= pai aj2'. This is correct because the right-hand side 
is the binary expansion of m. When m is negative, the leading 
bit a, of the one’s complement expansion of m is 1. The 
remaining n — | bits can be obtained by subtracting —m from 
111... 1 (where there are n — 1 1s), because subtracting a bit 
from | is the same as complementing it. Hence, the bit string 
Gn—2 ... do is the binary expansion of (2”~! — 1) — (—m). 
Solving the equation (2”~! — 1) — (—m) = ae a2! for 
m gives the desired equation because ad,-; = 1. 41. a) —7 
b) 13. c)—15 d)—1 43. To obtain the two’s complement 
representation of the sum of two integers, add their two’s 
complement representations (as binary integers are added) 
and ignore any carry out of the leftmost column. However, 
the answer is invalid if an overflow has occurred. This happens 
when the leftmost digits in the two’s complement representa- 
tion of the two terms agree and the leftmost digit of the answer 
differs. 45.Ifm > 0, then the leading bit a,_ is 0 and the 


formula reads m = ae aj2!. This is correct because the 


right-hand side is the binary expansion of m. If m < 0, its 
two’s complement expansion has | as its leading bit and the 
remaining n— 1 bits are the binary expansion of 2”—! — (—m). 
This means that (2!) — (—m) = Yo, aj2!. Solving for m 
gives the desired equation because d,_; = 1. 47. 4n 


49. procedure Cantor(x: positive integer) 


nee lif 

while (n+ 1) -f<x 
ni=n+1 
fi=fin 

yi x 

while n > 0 

an ‘= Ly/f J 
yi=y—an: f 
f= f/n 
n:i=n-—1 


{x = ann! + dy_1(n — I)!+---+ay1!} 


51. First step: c = 0,d = 0,59 = 1; second step: c = 0, 
d = 1, s; = 0; third step: c = 1, d = 1, s2 = 0; fourth step: 
c=1,d = 1,53 = 0; fifth step: c = 1,d = 1, s4 = 1; sixth 
step:c = 1,55 = 1 
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53. procedure subtract(a, b: positive integers, a > b, 
@ = (Gn—14n—2---41a0)2, 
b = (by—-1bn—2 . ..b1b0)2) 
B := 0 {B is the borrow} 
for j :=Oton—1 
ifa; = b; + B then 
sj i=aj—b;—B 


B:=0 

else 

Sj =aj+2—b;—-B 
B:=1 


{(Sp—15n—2 -.-518Q)2 18 the difference} 


55. procedure compare(a, b: positive integers, 
a = (Anan—1 .--41a0)2; 
b = (bnbn—1 ...b1b0)2) 
k:=n 
while ay = by andk > 0 
k:=k-1 
if a, = b, then print “a equals b” 
if ax > bx then print “a is greater than b” 
if a, < b,x then print “a is less than b” 


57. O(logn) 59. The only time-consuming part of the al- 
gorithm is the while loop, which is iterated g times. The work 
done inside is a subtraction of integers no bigger than a, which 
has log a bits. The result now follows from Example 9. 


Section 4.3 


1. 29,71,97 prime; 21,111,143 not prime 3.a)2?- 11 
b)2- 37-7 )3° d)7- 11-13 e)11- 101 f)2- 3°. 
5e7-19+37 8, 2°~37<8? <7 
7. procedure primetester(n : integer greater than 1) 
isprime := true 
d:=2 
while isprime and d < ./n 
ifn mod d = 0 then isprime := false 
else d :=d+1 
return isprime 


9. Write n = rs, wherer > 1 ands > 1. Then 2” —1 = 
ws _y= (2")s t= (2" 1)((2")§ 14 (27)s 24.(2")8 34 
.-++ 1). The first factor is at least 22 — 1 = 3 and the second 
factor is at least 27 + 1 = 5. This provides a factoring of 
2” — | into two factors greater than 1, so 2” — 1 is composite. 
11. Suppose that log, 3 = a/b where a,b € Z* andb F 0. 
Then 24/6 = 3, so 2% = 3°. This violates the fundamental 
theorem of arithmetic. Hence, log, 3 is irrational. 13. 3, 5, 
and 7 are primes of the desired form. 15.1, 7, 11, 13, 17, 
19, 23,29 17.a)Yes b)No c)Yes d)Yes_ 19. Suppose 
that n is not prime, so that n = ab, where a and b are inte- 
gers greater than 1. Because a > 1, by the identity in the hint, 
2° — 1 isa factor of 2” — | that is greater than 1, and the second 
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factor in this identity is also greater than 1. Hence, 2” — | is 
not prime. 21.a)2 b)4 c)12 23. ¢(p*) = p* — pk! 
25.a)3°-5° by c)23! d)41- 43-53 e)1 1111 
Boal eS WP « Be Be Pe Ms 
13-17 ¢) 233! d) 41-43-53 e) 2!23!35!77?! f) Undefined 
29. gcd (92928, 123552) = 1056; Icem(92928, 123552) = 
10,872,576; both products are 11,481,440,256. 31. Because 
min(x, y) + max(x, y) = x + y, the exponent of p; in 
the prime factorization of gcd(a, b) - lem(a, b) is the sum of 
the exponents of p; in the prime factorizations of a and b. 
33. a)6 b)3 ce) 11 d)3 e)40 f)12 35.9 37. By Exer- 
cise 36 it follows that ged(2 — 1, (2 — 1) mod (2? — 1)) = 
gced(2? — 1, 27™d> _ 1). Because the exponents involved 
in the calculation are b and a mod J, the same as the quan- 
tities involved in computing gcd(a, b), the steps used by the 
Euclidean algorithm to compute ged(2“ — 1, 2’ — 1) run in 
parallel to those used to compute gcd(a, b) and show that 
gcd(2* — 1,2? — 1) = 28°) _ 1, 39, all = 
(-1)- 10+ 1-11 byl = 21 .- 21 + (-10) - 44 
ec) 12 = (-1)-36+ 48 djl = 13-55 4 (—21) - 34 
e) 3 = 11-213+(—20)-117 f) 223 = 1-0+1-223 g) 1 =37- 
2347 + (—706) - 123 h) 2= 1128 - 3454 +(—835) - 4666 
i) 1 = 2468 -9999+(—2221)-11111 41. (—3)-264+1-91 = 
13. 43. 34- 144+ (—55)-89=1 


45. procedure extended Euclidean(a, b: positive integers) 


xi=a 
yi=b 
oldolds := | 
olds := 0 
oldoldt := 0 
oldt := 1 
while y 4 0 
q:=xdivy 
r:= x mod y 
xX:i=y 
vir 


s := oldolds — q - olds 
t := oldoldt — q - oldt 
oldolds := olds 
oldoldt := oldt 
olds := s 
oldt := t 
{ged(a, b) is x, and (oldolds)a + (oldoldt)b = x} 


47. a) dn = 1 ifn is prime and a, = 0 otherwise. b) dy is the 
smallest prime factor of n with a; = 1. ¢) a, is the number of 
positive divisors of n. d) a, = 1 ifn has no divisors that are 
perfect squares greater than | and a, = 0 otherwise. e) a, is 
the largest prime less than or equal ton. f) a, is the product of 
the first — 1 primes. 49. Because every second integer is 
divisible by 2, the product is divisible by 2. Because every third 
integer is divisible by 3, the product is divisible by 3. Therefore 
the product has both 2 and 3 in its prime factorization and is 
therefore divisible by3-2=6. 51.n = 1601 is acounterex- 
ample. 53 Setting k = a+b-+1 will produce the composite 
number a(a+b+1)+b =a?+ab+a+b = (at+l)(at+b). 


55. Suppose that there are only finitely many primes of the 
form 4k + 3, namely gj, 92, .--,; Gn, where qi = 3, q2 = 7, 
and so on. Let Q = 4q1q2 ++ dy — 1. Note that Q is of the form 
4k + 3 (where k = qiq2-+-qn — 1). If Q is prime, then we 
have found a prime of the desired form different from all those 
listed. If Q is not prime, then Q has at least one prime factor 
not in the list q1, g2,..., Yn, because the remainder when Q 
is divided by qj is g; — 1, and q; — 1 4 0. Because all odd 
primes are either of the form 4k + 1 or of the form 4k + 3, and 
the product of primes of the form 4k + 1 is also of this form 
(because (4k + 1)(4m-+ 1) = 4(4km+k+m)-+1), there must 
be a factor of Q of the form 4k + 3 different from the primes 
we listed. 57. Given a positive integer x, we show that there 
is exactly one positive rational number m/n (in lowest terms) 
such that K (m/n) = x. From the prime factorization of x, read 
off the m and n such that K (m/n) = x. The primes that occur 
to even powers are the primes that occur in the prime factor- 
ization of m, with the exponents being half the corresponding 
exponents in x; and the primes that occur to odd powers are 
the primes that occur in the prime factorization of n, with the 
exponents being half of one more than the exponents in x. 


Section 4.4 


1.15-7= 105 = 1(mod 26) 3.7 5. a)7 b)52 ce) 34 
d) 73/7. Suppose that b and c are both inverses of a modulo 
m. Then ba = 1 (mod m) and ca = 1 (mod m). Hence, 
ba = ca (mod m). Because gced(a, m) = 1 it follows by The- 
orem 7 in Section 4.3 thatb=c(mod m). 9.8 11. a) 67 
b) 88 c)146 13.3 and6 15.Let m’ = m/ gcd(c, m). 
Because all the common factors of m and c are divided out of 
m to obtain m’, it follows that m’ and c are relatively prime. 
Because m divides ac — bc = (a — b)c, it follows that m’ 
divides (a — b)c. By Lemma 3 in Section 4.3, we see that 
m’ divides a — b, soa = b(mod m’). 17. Suppose that 
x? = 1(mod p). Then p divides x2 — 1 = (x + 1)(x — 1). 
By Lemma 2 it follows that p | x + lor p | x — 1, so 
x = -1 (mod p) orx = 1(mod p). 19. a) Suppose that 
ia = ja (mod p), where 1 <i < j < p. Then p divides 
ja—ia =a(j —i). By Theorem 1, because a is not divisible 
by p, p divides 7 — i, which is impossible because j — i is 
a positive integer less than p. b) By part (a), because no two 
of a, 2a,..., (p — 1)a are congruent modulo p, each must be 
congruent to a different number from | to p—1. It follows that 
a-2a-3a-::--- (p—1)-a=1.-2-3.---- (p—1) (mod p). It 
follows that (p—1)!-a?~! = p—1 (mod p). ¢) By Wilson’s 
theorem and part (b), if p does not divide a, it follows that 
(—1)-a?—! = —1 (mod p). Hence, a?~! = 1 (mod p). d) If 
p|a,then p | a?. Hence, a? =a=0 (mod p). If p does not 
divide a, then a?—! = a (mod p), by part (c). Multiplying 
both sides of this congruence by a gives a? = a (mod p). 
21. All integers of the form 323 + 330k, where k is an integer 
23. All integers of the form 53 + 60k, where k is an integer 


25. procedure chinese(m,,m2,..., My : relatively 
prime positive integers ; a, d2,..., dy, : integers) 
m:=1 
for k := 1 ton 
m:=m-mr, 
for k := 1 ton 
My := m/mg 
Ve = M,' mod m, 
x:=0 
for k := 1ton 
x:= x +arMy yz 
while x > m 
xi=x—-—m 
return x {the smallest solution to the system 
{x = az (mod mz), k = 1,2,...,n }} 


27. All integers of the form 16 + 252k, where k is an inte- 
ger 29. Suppose that p is a prime appearing in the prime 
factorization of m 1m2---m,. Because the m;s are relatively 
prime, p is a factor of exactly one of the mj;s, say m;. Be- 
cause m; divides a — b, it follows that a — b has the fac- 
tor p in its prime factorization to a power at least as large 
as the power to which it appears in the prime factoriza- 
tion of m;. It follows that mjmz--- my, divides a — b, so 
a = b (mod mjm2:---m,). 31.x = 1 (mod 6) 33.7 
35.a?-2-a =a-a?-? =a?! =1 (mod p) 37. a) By 
Fermat’s little theorem, we have 2!9 = 1 (mod 11). Hence, 
2340 — (210)34 = 134 — 1 (mod 11). b) Because 32 = 1 
(mod 31), it follows that 2340 = (25) = 32° = 1% — ] 
(mod 31). c) Because 11 and 31 are relatively prime, and 

1-31 = 341, it follows by parts (a) and (b) and Exer- 
cise 29 that 24° = 1 (mod 341). 39.a)3, 4, 8 b) 983 
41. Suppose that g is an odd prime with g | 2?—1. By Fermat’s 
little theorem, g | 24 —-!_ 1. From Exercise 37 in Section 4.3, 
gcd(2? — 1, 29-! — 1) = 28°4(P-4-) _ |, Because q is acom- 
mon divisor of 2? —1 and 29~! —1, ged(2?—1, 24-!—1) > 1. 
Hence, gcd(p, g — 1) = p, because the only other possibility, 
namely, gcd(p, q—1) = 1, gives us ged(2? —1, 24-!—1) = 1. 
Hence, p | g — 1, and therefore there is a positive integer 
m such that g — 1 = mp. Because qg is odd, m must be 
even, say, m = 2k, and so every prime divisor of 2? — 1 
is of the form 2kp + 1. Furthermore, the product of num- 
bers of this form is also of this form. Therefore, all divisors 
of 2? — 1 are of this form. 43. My, is not prime; M17 
is prime. 45. First, 2047 = 23 - 89 is composite. Write 
2047 — 1 = 2046 = 2.- 1023, so s = 1 andt = 1023 in 
the definition. Then 2!073 = (2!!)93 = 70489 = 1% =1 
(mod 2047), as desired. 47. We must show that p2820 — 1 
(mod 2821) for all b relatively prime to 2821. Note that 
2821 = 7. 13 - 31, and if ged(b, 2821) = 1, then 
gcd(b, 7) = ged(b, 13) = gced(b, 31) = 1. Using Fermat’s lit- 
tle theorem we find that b° = 1 (mod 7), b!? = 1 (mod 13), 
and b*° = 1 (mod 31). It follows that 578° = (p°)47° = 1 
(mod 7), b2829 = (b!2)235 = 1 (mod 13), and 52820 = 
(b39)94 = 1 (mod 31). By Exercise 29 (or the Chinese re- 
mainder theorem) it follows that b282° = 1 (mod 2821), as 
desired. 49. a) If we multiply out this expression, we get 
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n = 1296m> + 396m? + 36m + 1. Clearly 6m |n — 1, 
12m |n—1, and 18m |n — 1. Therefore, the conditions of Ex- 
ercise 48 are met, and we conclude that n is a Carmichael 
number. b) Letting m = 51 givesn = 172,947,529. 
51.0 = (0,0), 1 = G,1).2 = @,.2);3 = (,.3), 
4= (1,4),5 = (2,0), 6 = (0, 1),7 = C1, 2), 8 = (2, 3), 
9= (0,4), 10= (1, 0), 11 = (2, 1), 12 = (0, 2), 13 = C1, 3), 
14 = (2,4) 53. We have my = 99, m2 = 98, m3 = 97, 
and m4 = 95,som = 99 - 98-97-95 = 89,403,930. We 
find that My = m/m, = 903,070, Mz = m/m2 = 912,285, 
M3 = m/m3z = 921,690, and My = m/m4 = 941,094. 
Using the Euclidean algorithm, we compute that y) = 37, 
yo = 33, y3 = 24, and yg = 4 are inverses of My, modulo 
m, fork = 1, 2, 3, 4, respectively. It follows that the solu- 
tion is 65 - 903,070 - 37 + 2- 912,285 - 33 + 51-921,690 - 
24+ 10 - 941,094-4 = 3,397,886,480 = 537,140 
(mod 89,403,930). 55.log,5 = 16, log,6 = 14 
57. log; 1 = 0, log; 2 = 14, log; 3 = 1, log;4 = 12, 
log3 5 = 5, log; 6 = 15, log; 7 = 11, log; 8 = 10, log; 9 = 2 
log; 10 = 3, log; 11 = 7, log; 12 = 13, log, 13 = 4, 
log; 14 = 9, log; 15 = 6, log3 16 = 8 59. Assume that s is 
a solution of x2 =a (mod p). Then because (—s)? — s’, - 

is also a solution. Furthermore, s # —s (mod p). Otherwise, 
p | 2s, which implies that p | s, and this implies, using 
the original assumption, that p | a, which is a contradiction. 
Furthermore, if s and ¢ are incongruent solutions modulo p, 
then because s* = t? (mod p), p | s? — t?. This implies that 
p|(s+t)(s —t), and by Lemma 3 in Section 4.3, p | s —t 
or p | s +t, so s = t (mod p) or s = —t (mod p). Hence, 
there are at most two solutions. 61. The value of (4) de- 
pends only on whether a is a quadratic residue modulo p, that 
is, whether w=a (mod p) has a solution. Because this de- 
pends only on the equivalence class of a modulo p, it follows 
that (4) = (4) ifa = b (mod p). 63. By Exercise 62, 
(2)(8)= aP—-D2p@R-D/2 = (apy P-Y/? = =(%) (mod p). 
65. x = 8, 13, 22, or 27 (mod 35) 67. Compute r® mod p 
fore = 0,1,2,..., p — 2 until we get the answer a. Worst 
case and average case time complexity are O(p log p). 


Section 4.5 


1. 91, 57, 21,5  3.a)7, or ae 18, 0 
available space mod 31. 1,5, 4, 1, 5, 4, 1, 5, 4, 

7.2, 6, 7, 10, 8, 2, 6, 7, is 8, tan 2357, 5554, 8469, 
7239, 4031, 2489, 1951,8064 11.2,1,1,1,... 13. Only 
string (d) 15.4 17.Correctly, of course 19. a) Not 
valid b) Valid c) Valid d) Notvalid 21.a)Nob)5c)7d)8 
23. Transposition errors involving the last digit 25. a) Yes 
b) No c)Yes d)No 27. Transposition errors will be de- 
tected if and only if the transposed digits are an odd num- 
ber of positions apart and do not differ by 5. 29. a) Valid 
b) Not valid c) Valid d) Valid 31. Yes, as long as the 
two digits do not differ by 7 33. a) Not valid b) Valid 
c) Valid d) Not valid 35. The given congruence is equiva- 
lent to 3d, + 4d2 + 5d3 + 6d4 + 7d5 + 8do + 9d7 + 10dg = 0 
(mod 11). Transposing adjacent digits x and y (with x on the 


b) Take the next 
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left) causes the left-hand side to increase by x — y. Because 
x #y (mod 11), the congruence will no longer hold. There- 
fore errors of this type are always detected. 


Section 4.6 


1.a) GRQRW SDVV JR b) QB ABG CNFFTB c) QX UXM 
AHJJ ZX  3.a) KOHQV MCIF GHSD_ b) RVBXP TJPZ 
NBZX c)DBYNE PHRM FYZA 5. a) SURRENDER 
NOW b)BE MY FRIEND c) TIME FOR FUN’ 7.TO 
SLEEP PERCHANCE TO DREAM’ 9.ANY SUFFI- 
CIENTLY ADVANCED TECHNOLOGY IS INDISTIN- 
GUISHABLE FROM MAGIC 11. p = 7c + 13 mod 26 
13.a = 18,b = 5 15. BEWARE OF MARTIANS 
17. Presumably something like an = affine cipher 
19. HURRICANE = 21. The length of the key may well be 
the greatest common divisor of the distances between the starts 
of the repeated string (or a factor of the gcd). 23. Suppose 
we know bothn = pq and (p—1)(q—1).To find p and q, first 
note that (p— 1)(¢—1) = pgq—p—q+1l=n-—(pt+q)+l. 
From this we can finds = p+ q. Becauseg = s — p, 
we haven = p(s — p). Hence, p* — ps +n = 0. We 
now can use the quadratic formula to find p. Once we have 
found p, we can find g because g = n/p. 25. 2545 2757 
1211 27.SILVER 29. Alice sends 5° mod 23 = 16 to 
Bob. Bob sends 5° mod 23 = 20 to Alice. Alice computes 
208 mod 23 = 6 and Bob computes 16° mod 23 = 6. The 
shared key is 6. 31. 2186 2087 1279 1251 0326 0816 1948 
33. Alice can decrypt the first part of Cathy’s message to 
learn the key, and Bob can decrypt the second part of Cathy’s 
message, which Alice forwarded to him, to learn the key. No 
one else besides Cathy can learn the key, because all of these 
communications use secure private keys. 


Supplementary Exercises 


1. The actual number of miles driven is 46518 + 100000k for 
some natural number k. 3.5, 22, —12, —29 5. Because 
ac = bc (mod m) there is an integer k such that ac = be + 
km. Hence, a — b = km/c. Because a — b is an integer, 
c | km. Letting d = gcd(m,c), write c = de. Because no 
factor of e divides m/d, it follows that d | m and e | k. Thus 
a—b = (k/e)(m/d), where k/e € Zand m/d € Z. There- 
forea = b(modm/d). 7. Proof of the contrapositive: 
If n is odd, then n = 2k + 1 for some integer k. Therefore 
n? +1 = (2k+ 1)? +1 = 4k? +4k +2 =2 (mod 4). But 
perfect squares of even numbers are congruent to 0 modulo 4 
(because (2m) = 4m), and perfect squares of odd numbers 
are congruent to 1 or 3 modulo 4, so n? + | is not a perfect 
square. 9. nis divisible by 8 if and only if the binary expan- 
sion of n ends with 000. 11. We assume that someone has 
chosen a positive integer less than 2”, which we are to guess. 
We ask the person to write the number in binary, using leading 
Os if necessary to make it n bits long. We then ask “Is the first 
bit a 1?’’, “Is the second bit a 1?”’, “Is the third bit a 1?”, and so 


on. After we know the answers to these n questions, we will 
know the number, because we will know its binary expansion. 
13. (€ndn—1---4140)10 = > -p—0 10°a, = reo A (mod 9) 
because 10 = 1 (mod 9) for every nonnegative integer k. 
15. Because for all k < n, when Q,, is divided by k the re- 
mainder will be 1, it follows that no prime number less than 
or equal to n is a factor of Q,,. Thus by the fundamental theo- 
rem of arithmetic, Q,, must have a prime factor greater than n. 
17. Takea = 10andb = 1inDirichlet’stheorem. 19. Every 
number greater than | 1 can be written as either 8-++-2n or 9+2n 
forsomen > 2. 21. Assume that every even integer greater 
than 2 is the sum of two primes, and let n be an integer greater 
than 5. If n is odd, write n = 3 + (n — 3) and decompose 
n—3 = p-+q into the sum of two primes; if 7 is even, then 
write n = 2 + (n — 2) and decompose n — 2 = p+ q into 
the sum of two primes. For the converse, assume that every 
integer greater than 5 is the sum of three primes, and let n be 
an even integer greater than 2. Write n + 2 as the sum of three 
primes, one of which is necessarily 2,son+2=2+p+q, 
whencen = p+q. _ 23. Recall that a nonconstant poly- 
nomial can take on the same value only a finite number of 
times. Thus f can take on the values 0 and +1 only finitely 
many times, so if there is not some y such that f() is com- 
posite, then there must be some xo such that + f (xo) is prime, 
say p. Look at f (xo + kp). When we plug x9 + kp in for x 
in the polynomial and multiply it out, every term will contain 
a factor of p except for the terms that form f (xq). Therefore 
f(xotkp) = f (xo) +mp = (m+1)p for some integer m. As 
k varies, this value can be 0, p, or — p only finitely many times; 
therefore it must be a composite number for some values of k. 
25.1 27.1 29. Ifnot, then suppose that gi, g2,..., Gn are 
all the primes of the form 6k + 5. Let Q = 6q1q2--- qn — 1. 
Note that Q is of the form 6k +5, where k = qig2--- qn —1. 
Let Q = p| p2--- p; be the prime factorization of Q. No p; is 
2, 3, or any g;, because the remainder when Q is divided by 2 
is 1, by 3 is 2, and by q; is g; — 1. All odd primes other than 3 
are of the form 6k + | or 6k + 5, and the product of primes of 
the form 6k + 1 is also of this form. Therefore at least one of 
the p;’s must be of the form 6k+5,acontradiction. 31. The 
product of numbers of the form 4k + 1 is of the form 4k + 1, 
but numbers of this form might have numbers not of this form 
as their only prime factors. For example, 49 = 4-12 + 1, but 
the prime factorization of 49 is 7-7 = (4-1+3)(4-1+ 3). 
33. a) Not mutually relatively prime b) Mutually relatively 
prime c) Mutually relatively prime d) Mutually relatively 
prime 351 37.x =28 (mod 30) 39. By the Chinese 
remainder theorem, it suffices to show that n? —n = 0 
(mod 2), n? —n = 0 (mod 3), and n? — n = 0 (mod 5). 
Each in turn follows from applying Fermat’s little theorem. 
41. By Fermat’s little theorem, p?—! = 1 (mod q) and clearly 
q?—! = 0 (mod q). Therefore p?-! + g?-! =14+0=1 
(mod q). Similarly, p?~! + g?~! = 1 (mod p). It follows 
from the Chinese remainder theorem that p?—! + g?~! = 1 
(mod pq). 43. Ifa; is changed from x to y, then the change 
in the left-hand side of the congruence is either y — x or 
3(y — x), modulo 10, neither of which can be 0 because | and 


3 are relatively prime to 10. Therefore the sum can no longer 
be 0 modulo 10. 45. Working modulo 10, solve for do. 
The check digit for 11100002 is 5. 47. PLEASE SEND 
MONEY 49. a) QAL HUVEM AT WVESGB b) QXB 
EVZZL ZEVZZRFS 


CHAPTER 5 


Section 5.1 


1. Let P(n) be the statement that the train stops at sta- 
tion n. Basis step: We are told that P(1) is true. Induc- 
tive step: We are told that P(n) implies P(n + 1) for each 
n > 1. Therefore by the principle of mathematical induc- 
tion, P(n) is true for all positive integers n. 3.a) 127 = 
1-2.-3/6 b) Both sides of P(1) shown in part (a) equal 1. 
ce) 12 +27 4+---+k? = kk + 1)(2k +1)/6 d) For each 
k > 1 that P(k) implies P(k + 1); in other words, that as- 
suming the inductive hypothesis [see part (c)] we can show 
P42?4---+hR + (kK +1? = (K+ DK + 2)(2k + 3)/6 
e) (17 +227 4+--- +k) + (k + 1)? = [kK + D(2k + 
1/6] + & + 1)? =[k + D/6MKQk + 1) + 6K + 
1] = [Kk + 1)/6]2k? + 7k + 6) = [(k + 1)/6](k + 
2)(2k+3) = (k+1)(K+2)(2k+3)/6 f) We have completed 
both the basis step and the inductive step, so by the principle 
of mathematical induction, the statement is true for every pos- 
itive integern. 5. Let P(n) be “12 +3?+---+(2n+1)? = 
(n+ 1)Qn + 1)(2n + 3)/3.” Basis step: P (0) is true because 
?=1= (0+1)(2-0+1)(2-04 3)/3. Inductive step: Assume 
that P(k) is true. Then 17 +3?+---4+(2k+1)?+[2k+1)+ 
1P = (K+ 1) Qk + 1)(2k +3)/3 + (2k +3)? = (2k 43) [(k 
1)(2k+1)/3+ (2k +3)] = (2k +3) (2k? +9k +10) /3 = (2k 
3)(2k+5)(k+2)/3 = (K+ 1) +H [2(k+1)4- 1[2(K4+-1)+3)/3. 
7. Let P(n) be ‘i 03° six sore — 1)/4.” Basis step: 
P(0) is true because eee si = 3 = 3(5! — 1)/4. 
Inductive step: Assume that >= qa°) = 35h 1)/4. 
Then 4453-5! = (hg 3-54) +3- SHH = 3654! — 
1)/443.54! = 3654 rea, 5k+1 _ 1) /4 = 3(5'*? — 1) /4, 
9. a) 24+4+4+6+4+---+2n =n(n+1) b) Basis step: 2 = 1-(1+1) 
is true. Inductive step: Assume that2++4+6+---+2k = 
k(k + 1). Then (2+ 4+6+---+ 2k) + 2(kK +1) = 
k(kK+1)+2(K+1) = (K+1)(K4+2). 11a) Lj=l 1/2/ = 

(2” — 1)/2” b) Basis step: P(1) is true because } 5 = (2)- 

1)/2!. Inductive step: Assume that )~* j= ies (ok 1) /2k. 


Then bea w= a (y= 1 a) a stn = x a stn 
wet = EEF 13. Let P(n) be “1? — 2? + 3? — 
» + (-1)"'n? = (-1)""! n(n + 1)/2.” Basis step: P(1) 
is true because 12 = 1 = (—1)91?. Inductive step: Assume 
that P(k) is true. Then 12 — 2? + 32 — .-. + (—1)h-1k? + 
(—1*(k +1)? = (-D* kk + 2/2 + (HDR + 1)? = 
(-1)* + Dl-k/24+ &+ DI = C1)*K+ DIK/2)+ = 
(—1)k (K+ 1) (k+2)/2. 15. Let P(n) be “1-2+2-3+4+---+ 
n(n+1) =n(n+1)(n+2)/3.” Basis step: P (1) is true because 


Answers to Odd-Numbered Exercises S-29 


1-2=2 = 1(1+1)(1+2)/3. Inductive step: Assume that P (k) 
is true. Then 1-2+2-3-+---+k(kK+1)+(kK+1)(kK+2) = [k(kK+ 
I(K+2)/3)+ (+ D4+2) = (K+ D4A+2)1K/3) + 0 = 


(kK+1)(K+2)(K+3)/3. 17. Let P(n) be the statement that 
44944344... +n4 =n(n+1)Qn+1)Gn?+3n—1)/30. 
P(1) is true because 1 - 2-3 -5/30 = 1. Assume that P(k) 
is true. Then (14 + 244 344---+ kh) +k + D* = 


k(k + I)(2k + I BK? + 3k — 1)/30 + (k + 1)* = [(k 
1) /30][k(2k + 1)(Bk? + 3k — 1) + 30k + 17] = [tk 
1) /30](6k*+ + 39k3 + 91k? + 89k + 30) = [(k + 1)/30](k + 
2)(2k + 3)[3(k + 1)? + 3(k + 1) — 1]. This demonstrates that 
P(k + 1)istrue. 19.a)1+4+ i <2- 5 b) This is true 
because 5/4 is less than 6/4. I+ 4 +--+ <2-f 
d) For each k > 2 that P(k) implies P(k + 1); in other words, 
we want to show that assuming the inductive hypothesis [see 
part (c)] we can show 1 4 i vee = { Try < i ET 


je dy , 1 4 1 
Oltare tet gp < *-k + ep 


i 1 k242k+1—k +k 1 
2 [i wp? | =2 [ k(k+1)2 |= - k(k+1)2— k(k+D2 
2 


Fel ~ keen? < 2 ea f) We have completed both 
the basis step and the inductive step, so by the principle of 
mathematical induction, the statement is true for every inte- 
ger n greater than 1. 21. Let P(n) be “2” > n?.” Basis 
step: P(5) is true because 2> = 32 > 25 = 5°. Induc- 
tive step: Assume that P(k) is true, that is, ok + k?. Then 
Ql 7.2K Se ke Ss ke 4k > kr 42k 4-1 = (kK +1) 
becausek > 4. 23. By inspection we find that the inequality 
2n +3 < 2” does not hold for n = 0, 1, 2, 3. Let P(n) be the 
proposition that this inequality holds for the positive integer n. 
P (4), the basis case, is true because 2-4+-3 = 11 < 16= Dh 
For the inductive step assume that P (k) is true. Then, by the in- 
ductive hypothesis, 2(k+1)+3 = (2k+3)+2 < 2k 49. But be- 
causek > 1, 242 < 242" = 2k+1 This shows that P(k-+1) 
istrue. 25.Let P(n) be“l+nh < A1+A)",h > -1” 
Basis step: P (0) is true because 1+0-h =1<1=(1+h)°. 
Inductive step: Assume 1+ kh < (+ h)*. Then because 
(1+h) > 0, 1+h)'*! = (+h) +h) > +h) +k) = 
L+k+DA+kh? > 14+ (k+ 1h. 27. Let P(n) be 

+1/f/n > 2(/n+ 1- Ba 


“T/J/T + 1/J/2+1/V3 +--- 
Basis step: P(1) is true because 1 > 2 (/2 — 1). Induc- 
tive step: Assume that P(k) is true. Then 1 + 1/V¥2+---+ 


I/Vk + /VRk+T > 2(VkFI-1) + 1/Vk +1. If we 
show that 2(/k+1—1) + I/Vk+1 > 2(vk+2-1), 
it follows that P(k + 1) is true. This inequality is equiv- 
alent to 2 (Vk +2 —Vk+1) < = 1/V7k+1, which is 
equivalent i 2 (Sk +2— Sk +1) (Jk +2+ Jk +1) < 
Vk A/Vk +I + Vk +2//k +1. This is equivalent to 
2 < 1+ JVJk+2//k+ 1, which is clearly true. 29. Let 
P(n) be “Hyn < 1 +n.” Basis step: P(O) is true be- 
cause Hy = Ay = 1 < 1+ 0. Inductive yo Assume 
that Hye < 14+ -k. Then Aye = Ayet all ee 7 = 


L+k+2 (ser) <1+k+1=14(k+1. 31. Basis 


step: 12 + 1 = 2 is divisible by 2. Inductive step: Assume 
the inductive hypothesis, that k? + k is divisible by 2. Then 
(kKAL? + (kK 4D) Hk +2k 414k 41 = (+h) 42(k+1), 
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the sum of a multiple of 2 (by the inductive hypothesis) and a 
multiple of 2 (by definition), hence, divisible by 2. 33. Let 
P(n) be “n> — n is divisible by 5.” Basis step: P(Q) is true 
because 03 — 0 = 0 is divisible by 5. Inductive step: As- 
sume that P(k) is true, that is, k> — 5 is divisible by 5. Then 
(k+1)?—(kK+1) = (k°+5k4+10K3+ 10k7+5k+1)—(k+1) = 
(k> — k) + 5(k* + 2k? + 2k? + k) is also divisible by 5, 
because both terms in this sum are divisible by 5. 35. Let 
P(n) be the proposition that (2n — 1)? — 1 is divisible by 
8. The basis case P(1) is true because 8 | 0. Now as- 
sume that P(k) is true. Because [(2(k + 1) — 1]? —1 = 
[(2k — 1)? — 1] + 8k, P(k + 1) is true because both terms on 
the right-hand side are divisible by 8. This shows that P(n) 
is true for all positive integers n, so m2 — 1 is divisible by 
8 whenever m is an odd positive integer. 37. Basis step: 
11'+14.127!-! = 1214-12 = 133 Inductive step: Assume the 
inductive hypothesis, that 11”+! + 127”! is divisible by 133. 
Then 11@+D+1 4 yQ2@+D—-1 = yy. git} 4 144.1271 = 
11-1174? 4 (114 133)- 127-1 = 11damt! + 1222-1) + 
133 - 127"-!. The expression in parentheses is divisible by 
133 by the inductive hypothesis, and obviously the second 
term is divisible by 133, so the entire quantity is divisible by 
133, as desired. 39. Basis step: A, C By tautologically im- 
plies that ‘a Aj ‘a= , B;. Inductive step: Assume the 
inductive hypothesis that if A; © B; for j = 1,2,...,k, 

then (Yj ,AZC Ae , Bj. We wankeyshowihakit Ay c = h, 
for j = 1,2,. k+ 1, then FY) Ay efi; By: letx 
be an arbitrary element of VN Aj [ie 1 Aj) O Ags. 

Because x € ae j= , Aj, we know by ‘the dicuetive hypothe- 


sis that x € al _, Bj; because x € Ax41, we know from 
the given fact that Agit © Bryi that x € B41. There- 
fore, x € (Mit By) )O Bes = (Vs Bj. 41. Let P(n) be 
“(A{UA2U:+-UAn)NB = (AN B)U(A2N BYU: -U(An A 
B).’ Basis step: P (1) is trivially true. Inductive step: Assume 
that P(x) is true. Then (Aj U Az U--- UApU Ag JN B= 


[(Ay UA2U---UAR)U Aggy i JAB = (AU A2U---U AZ) 
BV VU (Agi OB) = (A, N B)U(A2N B) U---U(ARN 
B)| U (Aggi ON B) = (Ai 9 B) U (A219 B) U---U (ARN 


B)U (Ag41NB). 43. Let P(n) be “Ug_) Ak = ey Ak” 

Basis step: P(1) is trivially true. Inductive step: Assume that 
) tly. TA k 

P(k) is true. Then (j= a = (Usa A i) U Age 


(Ui Aj) N Am = (Mk 4j) NAc = (Vet Ay. 

45. Let P(n) be the statement that a set with n elements has 
n(n — 1)/2 two-element subsets. P (2), the basis case, is true, 
because a set with two elements has one subset with two 
elements—namely, itself—and 2(2 — 1)/2 = 1. Now as- 
sume that P(k) is true. Let S be a set with k + 1 elements. 
Choose an element a in S and let T = S — {a}. A two-element 
subset of S$ either contains a or does not. Those subsets not 
containing a are the subsets of T with two elements; by the 
inductive hypothesis there are k(k — 1)/2 of these. There are k 
subsets of S with two elements that contain a, because such a 
subset contains a and one of the k elements in 7. Hence, there 
are k(k—1)/2+k = (k+1)k/2 two-element subsets of $. This 


a 


completes the inductive proof. 47. Reorder the locations if 
necessary so that x) < x2 < x3 < +--+ < XxXq. Place the first 
tower at position fy = x; + 1. Assume tower k has been placed 
at position t;. Then place tower k + 1 at position th, = x +1, 
where x is the smallest x; greater than t, +1. 49. The two 
sets do not overlap ifn + 1 = 2. In fact, the conditional state- 
ment P(1) — P(2)is false. 51. The mistake is in applying 
the inductive hypothesis to look at max(x — 1, y— 1), because 
even though x and y are positive integers, x — | and y — 1 
need not be (one or both could be 0). 53. For the basis step 
(n = 2) the first person cuts the cake into two portions that she 
thinks are each 1/2 of the cake, and the second person chooses 
the portion he thinks is at least 1/2 of the cake (at least one of 
the pieces must satisfy that condition). For the inductive step, 
suppose there are k + 1 people. By the inductive hypothesis, 
we can suppose that the first k people have divided the cake 
among themselves so that each person is satisfied that he got 
at least a fraction 1/k of the cake. Each of them now cuts his 
or her piece into k+ 1 pieces of equal size. The last person gets 
to choose one piece from each of the first k people’s portions. 
After this is done, each of the first k people is satisfied that 
she still has (1/k)(k/(k + 1)) = 1/(k + 1) of the cake. To 
see that the last person is satisfied, suppose that he thought 
that the 7th person (1 < i < k) had a portion p; of the 
cake, where — Pi = 1. By choosing what he thinks is the 
largest piece from each person, he is satisfied that he has at 
least _, pi/(kK+ 1) = A/k +1) 2, pi = 1/k +1) of 
the cake. 55. We use the notation (i, 7) to mean the square 
in rowi and column j and use induction oni + j to show that 
every square can be reached by the knight. Basis step: There 
are six base cases, for the cases wheni + j < 2. The 
knight is already at (0, 0) to start, so the empty sequence of 
moves reaches that square. To reach (1, 0), the knight moves 
(0,0) > (2, 1) > (0,2) > (1, 0). Similarly, to reach (0, 1), 
the knight moves (0,0) — (1,2) > (2,0) > (0, 1). Note 
that the knight has reached (2, 0) and (0, 2) in the process. 
For the last basis step there is (0,0) — (1,2) — (2,0) > 
(0,1) > (2,2) > (0,3) > C, 1). Inductive step: Assume 
the inductive hypothesis, that the knight can reach any square 
(i, j) for which i + 7 = k, where k is an integer greater 
than 1. We must show how the knight can reach each square 
(i, j) wheni + j =k +1. Because k + 1 > 3, at least one 
of i and j; is at least 2. If i > 2, then by the inductive hypoth- 
esis, there is a sequence of moves ending at (i — 2, 7 + 1), 
becausei —-2+ 7+ 1 = i+ j-1 = k; from there 
it is just one step to (i, j); similarly, if 7 > 2. 57. Basis 
step: The base casesn = O andn = | are true because 
the derivative of x° is 0 and the derivative of x! = x is 1. 
Inductive step: Using the product rule, the inductive hypoth- 
esis, and the basis step shows that 4 xhtl = as x)= 
xe oxktxk dy = yx. kxk-l 4 xk a2 kxk xk = (k+ 1x8. 

59. Basis step: Fork = 0,1 = 1 aod m). Inductive step: 
Suppose that a = b (mod m) and ak = bk (mod m); we 
must show that gk! = bk+! (mod m). By Theorem 5 from 
Section 4.1, a - ak = b. bk (mod m), which by defini- 


tion says that att! = p+! (mod m). 61. Let P(n) be 
“[(p1 > p2) A (pz > p3) A +++ A (Pa-1 > Pn)| > 
[(pi A +++ A pn-1) > Pn.” Basis step: P (2) is true because 
(p1 > p2) > (pi — pz) is a tautology. Inductive step: 
Assume P(k) is true. To show [(p1 > p2) A ++: A (pe-1 > 
Pk) \ (Pk > Pe+id) > (Pi A+++ A pr-1 A Pk) > Prt] 
is a tautology, assume that the hypothesis of this conditional 
statement is true. Because both the hypothesis and P(k) are 
true, it follows that (py A --: A pe-1) — px is true. Be- 
cause this is true, and because py — px+1 is true (it is part 
of the assumption) it follows by hypothetical syllogism that 
(pi A+++ A pr-1) — pr+i is true. The weaker statement 
(pi A+++ A pe-1 A Pk) > pr+1 follows from this. 63. We 
will first prove the result when n is a power of 2, that is, if 
n = 2*,k = 1,2,.... Let P(k) be the statement A > G, 
where A and G are the arithmetic and geometric means, re- 
spectively, of a set of n = 2* positive real numbers. Basis 
step: k = 1andn = 2! = 2. Note that (,/a; — az)? > 0. 
Expanding this shows that a; — 2,/ajaz + az > 0, that is, 
(ay +a2)/2 > (ayar)'/2. Inductive step: Assume that P(k) is 
true, with n = 2*. We will show that P(k + 1) is true. We have 
2k+1 — In. Now (a1 + a2 + +++ + G2n)/(2n) = [(ai +. a2 + 

~+ an) /n + (Qn41 + an42 +-+++€2n)/n)/2 and similarly 
(a\a2 °°: a2) 1/2”) =[(a--: al (Qn4i-°° aay) 7. To 
simplify the notation, let A(x, y,...) and G(x, y,...) denote 
the arithmetic mean and geometric mean of x, y, ..., respec- 
tively. Also, if x < x’, y < y’, and so on, then A(x, y,...) < 
A(x’, y’,...) and G(x, y,...) < G(x’, y’,...). Hence, 
A(ay,..., 42) = A(A(Q,-.-, Gn), ACGn4i, +++, @n)) = 
A(G(a1,...,4n), G(an41, .++,@n)) = G(G(a, ..., an), 
G(adn41, --., 42n)) = G(ai,..., Gn). This finishes 
the proof for powers of 2. Now if n is not a power 
of 2, let m be the next higher power of 2, and let 
An+1, +--+ 4m all equal A(aj,...,a,) = a. Then we 


have [(a}a2---a,)a"—"|!/" < A(ay,..., Gm), because m is 
a power of 2. Because A(daj,..., Qn) = @, it follows 
that (aj -++ay)'/m™ql-n/™ < qm, Raising both sides to the 
(m/n)th power gives G(a1,..., dn) < A(@1,...,d). 
65. Basis step: For n = 1, the left-hand side is just 1. which 
is 1. Forn = 2, there are three nonempty subsets {1}, {2}, 
and {1, 2}, so the left-hand side is + + 5 + zy = 2. Inductive 
step: Assume that the statement is true for k. The set of the 
first k + 1 positive integers has many nonempty subsets, but 
they fall into three categories: a nonempty subset of the first 
k positive integers together with k + 1, anonempty subset of 
the first k positive integers, or just {k + 1}. By the inductive 
hypothesis, the sum of the first category is k. For the second 
category, we can factor out 1/(k + 1) from each term of the 
sum and what remains is just k by the inductive hypothesis, 
so this part of the sum is k/(k + 1). Finally, the third cate- 
gory simply yields 1/(k + 1). Hence, the entire summation 
isk+k/kK+1)+1/A +1) =k+1. 67. Basis step: 
If Ay C Apo, then A, satisfies the condition of being a sub- 
set of each set in the collection; otherwise Az C A}, so A2 
satisfies the condition. Inductive step: Assume the inductive 
hypothesis, that the conditional statement is true for k sets, 
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and suppose we are given k + | sets that satisfy the given 
conditions. By the inductive hypothesis, there must be a set 
A; for some i < k such that Aj C A; for! < j < k. 
If A; C Axg4i, then we are done. Otherwise, we know that 
Ax41 © Aj, and this tells us that Az+1 satisfies the condition 
of being a subset of Aj for! <j <k+1. 69.G(1)=0, 
G(2) = 1, G(3) = 3, G44) = 4_ 71. To show that 2n — 4 
calls are sufficient to exchange all the gossip, select persons 1, 
2, 3, and 4 to be the central committee. Every person outside 
the central committee calls one person on the central commit- 
tee. At this point the central committee members as a group 
know all the scandals. They then exchange information among 
themselves by making the calls 1-2, 3-4, 1-3, and 2-4 in that 
order. At this point, every central committee member knows 
all the scandals. Finally, again every person outside the central 
committee calls one person on the central committee, at which 
point everyone knows all the scandals. [The total number of 
calls is (n — 4) +4+ (n — 4) = 2n — 4.] That this cannot be 
done with fewer than 2n — 4 calls is much harder to prove; see 
Sandra M. Hedetniemi, Stephen T. Hedetniemi, and Arthur L. 
Liestman, “A survey of gossiping and broadcasting in com- 
munication networks,” Networks 18 (1988), no. 4, 319-349, 
for details. 73. We prove this by mathematical induction. 
The basis step (n = 2) is true tautologically. Forn = 3, 
suppose that the intervals are (a, b), (c, d), and (e, f), where 
without loss of generality we can assume thata < c < e. 
Because (a, b)N(e, f) #4 ¥, we must have e < b; fora similar 
reason, e < d. It follows that the number halfway between e 
and the smaller of b and d is common to all three intervals. 
Now for the inductive step, assume that whenever we have k 
intervals that have pairwise nonempty intersections then there 
is a point common to all the intervals, and suppose that we are 
given intervals ;, I2,..., J,+, that have pairwise nonempty 
intersections. For each7 from | to k, let Jj = 1) O Ip4 1. We 
claim that the collection Jj, J2,..., J, satisfies the inductive 
hypothesis, that is, that Jj, M Jj, 4 % for each choice of sub- 
scripts 7; and iz. This follows from the n = 3 case proved 
above, using the sets /;,, J;,, and 7,41. We can now invoke the 
inductive hypothesis to conclude that there is a number com- 
mon to all of the sets J; fori = 1, 2,...,k, which perforce 
is in the intersection of all the sets J; fori = 1,2,...,k +1. 
75. Pair up the people. Have the people stand at mutually dis- 
tinct small distances from their partners but far away from 
everyone else. Then each person throws a pie at his or her 
partner, so everyone gets hit. 


lnitea1 
SE 
oye 


79. Let P(n) be the statement that every 2” x 2” x 2” checker- 
board with a 1 x 1 x 1 cube removed can be covered by tiles 


oe 
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that are 2 x 2 x 2 cubes each with a 1 x 1 x 1 cube removed. 
The basis step, P(1), holds because one tile coincides with the 
solid to be tiled. Now assume that P (k) holds. Now consider a 
IFT OFT y OK+1 cube with a 1 x 1 x 1 cube removed. Split 
this object into eight pieces using planes parallel to its faces 
and running through its center. The missing | x 1 x 1 piece 
occurs in one of these eight pieces. Now position one tile with 
its center at the center of the large object so that the missing 
1 x 1 x 1 cube lies in the octant in which the large object is 
missing a 1 x 1 x 1 cube. This creates eight 2* x 2 x 2 cubes, 
each missing a 1 x 1 x 1 cube. By the inductive hypothesis 
we can fill each of these eight objects with tiles. Putting these 
tilings together produces the desired tiling. 

81. 


83. Let O(n) be P(n+b—1). The statement that P(n) is true 
forn = b,b+1,b+2,...is the same as the statement that 
Q(m) is true for all positive integers m. We are given that P(b) 
is true [i.e., that Q(1) is true], and that P(k) > P(k + 1) 
for allk > b [xe., that Q(m) — Q(m + 1) for all posi- 
tive integers m]. Therefore, by the principle of mathematical 
induction, Q(m) is true for all positive integers m. 


Section 5.2 


1. Basis step: We are told we can run one mile, so P(1) is 
true. Inductive step: Assume the inductive hypothesis, that we 
can run any number of miles from | to k. We must show that 
we can run k + 1| miles. If k = 1, then we are already told 
that we can run two miles. If k > 1, then the inductive hy- 
pothesis tells us that we can run k — | miles, so we can run 
(k-—1)+2=k+I1 miles. 3.a) P(8) is true, because we 
can form 8 cents of postage with one 3-cent stamp and one 
5-cent stamp. P(9) is true, because we can form 9 cents of 
postage with three 3-cent stamps. P(10) is true, because we 
can form 10 cents of postage with two 5-cent stamps. b) The 
statement that using just 3-cent and 5-cent stamps we can form 
j cents postage for all j with 8 < j < k, where we assume 
that k > 10 c) Assuming the inductive hypothesis, we can 
form k + | cents postage using just 3-cent and 5-cent stamps 
d) Because k > 10, we know that P(k — 2) is true, that is, that 
we can form k —2 cents of postage. Put one more 3-cent stamp 
on the envelope, and we have formed k + | cents of postage. 
e) We have completed both the basis step and the inductive 
step, so by the principle of strong induction, the statement is 
true for every integer n greater than or equal to 8. 5. a) 4, 
8, 11, 12, 15, 16, 19, 20, 22, 23, 24, 26, 27, 28, and all values 
greater than or equal to 30 _b) Let P(n) be the statement that 


we can form n cents of postage using just 4-cent and 11-cent 
stamps. We want to prove that P(m) is true for alln > 30. 
For the basis step, 30 = 11 + 11 + 4+ 4. Assume that 
we can form k cents of postage (the inductive hypothesis); 
we will show how to form k + 1 cents of postage. If the k 
cents included an 11-cent stamp, then replace it by three 4- 
cent stamps. Otherwise, k cents was formed from just 4-cent 
stamps. Because k > 30, there must be at least eight 4-cent 
stamps involved. Replace eight 4-cent stamps by three 1 1-cent 
stamps, and we have formed k + 1 cents in postage. c) P(n) 
is the same as in part (b). To prove that P(n) is true for all 
n > 30, we check for the basis step that 30 = 114+ 11+4+4, 
31 = 114444444444, 32 =4+44+44+44444+444, 
and 33 = 11+ 11+ 11. For the inductive step, assume the in- 
ductive hypothesis, that P(/) is true for all j with 30 < j <k, 
where k is an arbitrary integer greater than or equal to 33. We 
want to show that P(k + 1) is true. Because k — 3 > 30, we 
know that P(k — 3) is true, that is, that we can form k — 3 
cents of postage. Put one more 4-cent stamp on the envelope, 
and we have formed k + | cents of postage. In this proof, our 
inductive hypothesis was that P(j) was true for all values of 
j between 30 and k inclusive, rather than just that P (30) was 
true. 7. Wecan form all amounts except $1 and $3. Let P(n) 
be the statement that we can form n dollars using just 2-dollar 
and 5-dollar bills. We want to prove that P() is true for all 
n > 5. (It is clear that $1 and $3 cannot be formed and that 
$2 and $4 can be formed.) For the basis step, note that 5 = 5 
and 6 = 2+2-+2. Assume the inductive hypothesis, that P(j) 
is true for all 7 with 5 < j < k, where k is an arbitrary integer 
greater than or equal to 6. We want to show that P(k + 1) is 
true. Because k— 1 > 5, we know that P(k — 1) is true, that is, 
that we can form k — | dollars. Add another 2-dollar bill, and 
we have formed k+ 1 dollars. 9. Let P(n) be the statement 
that there is no positive integer b such that /2 = n/b. Basis 
step: P(1) is true because /2 > 1 > 1/b for all positive 
integers b. Inductive step: Assume that P(j) is true for all 
J < k, where k is an arbitrary positive integer; we prove that 
P(k+1) is true by contradiction. Assume that V2 = (k+ 1)/b 
for some positive integer b. Then 2b? = (k+ 1)*, so (k + 1)? 
is even, and hence, k + 1 is even. So write k + 1 = 2r for 
some positive integer t, whence 2b? = 4¢? and b? = 217. By 
the same reasoning as before, b is even, so b = 2s for some 
positive integer s. Then /2 = (k + 1)/b = (2t)/(2s) = t/s. 
But t < k, so this contradicts the inductive hypothesis, and 
our proof of the inductive step is complete. 11. Basis step: 
There are four base cases. Ifn = 1 = 4-0+1, then clearly the 
second player wins. If there are two, three, or four matches 
(n = 4-0+2,n = 4-0+3, orn = 4-1), then the first player can 
win by removing all but one match. Inductive step: Assume 
the strong inductive hypothesis, that in games with k or fewer 
matches, the first player can win if k = 0,2, or 3 (mod 4) and 
the second player can winif k = 1 (mod 4). Suppose we have 
a game with k + 1 matches, withk > 4. Ifk+1=0 (mod 4), 
then the first player can remove three matches, leaving k — 2 
matches for the other player. Because k — 2 = 1 (mod 4), by 
the inductive hypothesis, this is a game that the second player 


at that point (who is the first player in our game) can win. Sim- 
ilarly, if k + 1 = 2 (mod 4), then the first player can remove 
one match; and if k + 1 = 3 (mod 4), then the first player 
can remove two matches. Finally, if k + 1 = 1 (mod 4), then 
the first player must leave k, k — 1, or k — 2 matches for the 
other player. Because k = 0 (mod 4), k — 1 = 3 (mod 4), 
and k — 2 = 2 (mod 4), by the inductive hypothesis, this is 
a game that the first player at that point (who is the second 
player in our game) can win. 13. Let P(n) be the statement 
that exactly n — 1 moves are required to assemble a puzzle 
with n pieces. Now P(1) is trivially true. Assume that P(/) 
is true for all 7 < k, and consider a puzzle with k + 1 pieces. 
The final move must be the joining of two blocks, of size j 
and k + 1 — 7 for some integer j with 1 < 7 < k. By the 
inductive hypothesis, it required 7 — 1 moves to construct the 
one block, and k + 1 — j — 1 =k — j moves to construct the 
other. Therefore, 1+ (j — 1) + (k—j) = k moves are required 
inall,so P(k+1)istrue. 15. Let the Chomp board have n 
rows and n columns. We claim that the first player can win the 
game by making the first move to leave just the top row and 
leftmost column. Let P(n) be the statement that if a player 
has presented his opponent with a Chomp configuration con- 
sisting of just n cookies in the top row and n cookies in the 
leftmost column, then he can win the game. We will prove 
Vn P(n) by strong induction. We know that P(1) is true, be- 
cause the opponent is forced to take the poisoned cookie at 
his first turn. Fix k > 1 and assume that P(/) is true for all 
J < k. We claim that P(k + 1) is true. It is the opponent’s 
turn to move. If she picks the poisoned cookie, then the game 
is over and she loses. Otherwise, assume she picks the cookie 
in the top row in column j, or the cookie in the left column in 
row j, for some j with2 < 7 < k+ 1. The first player now 
picks the cookie in the left column in row j, or the cookie in 
the top row in column /, respectively. This leaves the position 
covered by P(j — 1) for his opponent, so by the inductive hy- 
pothesis, he can win. 17. Let P(n) be the statement that if a 
simple polygon with n sides is triangulated, then at least two 
of the triangles in the triangulation have two sides that border 
the exterior of the polygon. We will prove Vn > 4 P(n). The 
statement is clearly true for n = 4, because there is only one 
diagonal, leaving two triangles with the desired property. Fix 
k > 4 and assume that P(/) is true for all 7 with 4 < j <k. 
Consider a polygon with k + 1 sides, and some triangulation 
of it. Pick one of the diagonals in this triangulation. First sup- 
pose that this diagonal divides the polygon into one triangle 
and one polygon with k sides. Then the triangle has two sides 
that border the exterior. Furthermore, the k-gon has, by the 
inductive hypothesis, two triangles that have two sides that 
border the exterior of that k-gon, and only one of these trian- 
gles can fail to be a triangle that has two sides that border the 
exterior of the original polygon. The only other case is that 
this diagonal divides the polygon into two polygons with j 
sides and k + 3 — j sides for some j with4 < j <k—-1. 
By the inductive hypothesis, each of these two polygons has 
two triangles that have two sides that border their exterior, and 
in each case only one of these triangles can fail to be a trian- 
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gle that has two sides that border the exterior of the original 
polygon. 19.Let P(n) be the statement that the area of a 
simple polygon with n sides and vertices all at lattice points 
is given by /(P) + B(P)/2 — 1. We will prove P(n) for all 
n > 3. We begin with an additivity lemma: If P is a simple 
polygon with all vertices at lattice points, divided into poly- 
gons P; and P2 by a diagonal, then /(P) + B(P)/2-—1 = 
[1(P1) + B(P1)/2 — 1] + [1(P2) + B(P2)/2 — 1]. To prove 
this, suppose there are k lattice points on the diagonal, not 
counting its endpoints. Then /(P) = (P|) + 1(P2) +k and 
B(P) = B(P,) + B(P2) — 2k — 2; and the result follows 
by simple algebra. What this says in particular is that if Pick’s 
formula gives the correct area for P; and P>, then it must give 
the correct formula for P, whose area is the sum of the areas 
for P, and P); and similarly if Pick’s formula gives the correct 
area for P and one of the P;’s, then it must give the correct 
formula for the other P;. Next we prove the theorem for rect- 
angles whose sides are parallel to the coordinate axes. Such a 
rectangle necessarily has vertices at (a, b), (a,c), (d, b), and 
(d, c), where a, b, c, andd are integers with b < canda <d. 
Its area is (c — b)(d — a). Also, B = 2(c —b + d — a) and 
I = (c—b—1)(d—a-—1) = (c—b)(d—a)—(c—b)—(d—a)+1. 
Therefore, J] + B/2—1=(c—b)(d —a) —(c—b)—-(d 

a)+1+(c—b+d-—a) —1 = (c —b)(d — a), whichis 
the desired area. Next consider a right triangle whose legs are 
parallel to the coordinate axes. This triangle is half a rectangle 
of the type just considered, for which Pick’s formula holds, so 
by the additivity lemma, it holds for the triangle as well. (The 
values of B and J are the same for each of the two triangles, 
so if Picks’s formula gave an answer that was either too small 
or too large, then it would give a correspondingly wrong an- 
swer for the rectangle.) For the next step, consider an arbitrary 
triangle with vertices at lattice points that is not of the type al- 
ready considered. Embed it in as small a rectangle as possible. 
There are several possible ways this can happen, but in any 
case (and adding one more edge in one case), the rectangle 
will have been partitioned into the given triangle and two or 
three right triangles with sides parallel to the coordinate axes. 
Again by the additivity lemma, we are guaranteed that Pick’s 
formula gives the correct area for the given triangle. This com- 
pletes the proof of P (3), the basis step in our strong induction 
proof. For the inductive step, given an arbitrary polygon, use 
Lemma | in the text to split it into two polygons. Then by 
the additivity lemma above and the inductive hypothesis, we 
know that Pick’s formula gives the correct area for this poly- 
gon. 21. a) In the left figure Zabp is smallest, but bp is not 
an interior diagonal. b) In the right figure bd is not an interior 
diagonal. c) In the right figure bd is not an interior diagonal. 
23. a) When we try to prove the inductive step and find a tri- 
angle in each subpolygon with at least two sides bordering the 
exterior, it may happen in each case that the triangle we are 
guaranteed in fact borders the diagonal (which is part of the 
boundary of that polygon). This leaves us with no triangles 
guaranteed to touch the boundary of the original polygon. 
b) We proved the stronger statement Vn > 4 T (n) in Exercise 
17. 25. a) The inductive step here allows us to conclude that 
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P(3), P(5), ... areall true, but we can conclude nothing about 
P(2), P(4),.... b) P(m) is true for all positive integers n, us- 
ing strong induction. c) The inductive step here enables us to 
conclude that P(2), P(4), P(8), P(16), ...are all true, but we 
can conclude nothing about P(n) when n is not a power of 2. 
d) This is mathematical induction; we can conclude that P (n) 
is true for all positive integers n. 27. Suppose, for a proof 
by contradiction, that there is some positive integer n such that 
P(n) is not true. Let m be the smallest positive integer greater 
than n for which P(m) is true; we know that such an m exists 
because P(m) is true for infinitely many values of m. But we 
know that P(m) — P(m—1),so P(m—1) is also true. Thus, 
m — 1 cannot be greater than n, som — 1 =n and P(n) isin 
fact true. This contradiction shows that P(n) is true for all n. 
29. The error is in going from the base case n = 0 to the next 
case, n = 1; we cannot write 1 as the sum of two smaller 
natural numbers. 31. Assume that the well-ordering prop- 
erty holds. Suppose that P(1) is true and that the conditional 
statement [P(1) A P(2)A---A P(n)] > P(n+1) is true for 
every positive integer n. Let S be the set of positive integers 
n for which P(n) is false. We will show S = %. Assume that 
S # W. Then by the well-ordering property there is a least 
integer m in S. We know that m cannot be | because P(1) is 
true. Because n = m is the least integer such that P (77) is false, 
P(1), P(2),..., P(m — 1) are true, and m — 1 > 1. Because 
[P() A P(2)A---A P(m — 1)] > P(m) is true, it follows 
that P(m) must also be true, which is a contradiction. Hence, 
S=9%. 33. Ineach case, give a proof by contradiction based 
ona “smallest counterexample,” that is, values of n and k such 
that P(n, k) is not true and n and k are smallest in some sense. 
a) Choose a counterexample with n + k as small as possible. 
We cannot have n = | andk = 1, because we are given 
that P(1, 1) is true. Therefore, either n > 1 ork > 1. In the 
former case, by our choice of counterexample, we know that 
P(n—1,k) is true. But the inductive step then forces P(n, k) 
to be true, a contradiction.The latter case is similar. So our 
supposition that there is a counterexample mest be wrong, 
and P(n, k) is true in all cases. b) Choose a counterexample 
with n as small as possible. We cannot have n = 1, because 
we are given that P(1, k) is true for all k. Therefore, n > 1. 
By our choice of counterexample, we know that P(n — 1, k) 
is true. But the inductive step then forces P(n, k) to be true, 
acontradiction. ¢) Choose a counterexample with k as small 
as possible. We cannot have k = 1, because we are given that 
P(n, 1) is true for all n. Therefore, k > 1. By our choice of 
counterexample, we know that P(n, k — 1) is true. But the 
inductive step then forces P(n, k) to be true, a contradiction. 
35. Let P(n) be the statement that if x1, x2,..., X, aren dis- 
tinct real numbers, then n — | multiplications are used to find 
the product of these numbers no matter how parentheses are 
inserted in the product. We will prove that P(7) is true using 
strong induction. The basis case P(1) is true because 1—1 = 0 
multiplications are required to find the product of x;, a product 
with only one factor. Suppose that P(x) is true for 1 <k <n. 
The last multiplication used to find the product of the n + 1 
distinct real numbers x1, x2, ..., Xn, Xn+1 18 a multiplication 


of the product of the first k of these numbers for some k and 
the product of the last n + 1 — k of them. By the inductive 
hypothesis, k — 1 multiplications are used to find the product 
of k of the numbers, no matter how parentheses were inserted 
in the product of these numbers, and n — k multiplications are 
used to find the product of the other n + 1 — k of them, no 
matter how parentheses were inserted in the product of these 
numbers. Because one more multiplication is required to find 
the product of all n + 1 numbers, the total number of multi- 
plications used equals (k — 1) + (n —k) + 1 = n. Hence, 
P(n+1)istrue. 37.Assume thata = dq+r=dq' +r’ 
with 0 < r <dand0 <r’ < d.Thend(g —q') =r’ —r. 
It follows that d divides r’ — r. Because —d < r’ —r < d, 
we have r’ — r = 0. Hence, r’ = r. It follows that g = q’. 
39. This is a paradox caused by self-reference. The answer is 
clearly “no.” There are a finite number of English words, so 
only a finite number of strings of 15 words or fewer; there- 
fore, only a finite number of positive integers can be so de- 
scribed, not allofthem. 41. Suppose that the well-ordering 
property were false. Let S be a nonempty set of nonnegative 
integers that has no least element. Let P(n) be the statement 
“i ¢ S fori =0,1,...,n.” P(O) is true because if 0 € S then 
S has a least element, namely, 0. Now suppose that P(n) is 
true. Thus, 0 ¢ S,1¢S,...,n ¢ S. Clearly, n+ 1 cannot be 
in S, for if it were, it would be its least element. Thus P(n + 1) 
is true. So by the principle of mathematical induction, n ¢ S 
for all nonnegative integers n. Thus, S = @, a contradiction. 
43. Strong induction implies the principle of mathematical in- 
duction, for if one has shown that P(k) — P(k + 1) is true, 
then one has also shown that [P(1)A---AP(k)] ~ P(k+1)is 
true. By Exercise 41, the principle of mathematical induction 
implies the well-ordering property. Therefore by assuming 
strong induction as an axiom, we can prove the well-ordering 


property. 
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loa) fC) =3, FQ) =5, f3) =7, f(4) =9 b) fC) = 3, 
FQ) = 9, f3) = 27, f(A) = 81 ©) fC) = 2, FQ) =4, 
f(3) = 16, f(4) = 65,536 d) fl) = 3, f(2) = 13, f3) = 
183, f (4) = 33,673 3. a) f(2) = —1, f(3) =5, f(4) =2, 
(5) =17b) f(2) = —4, f (3) = 32, f(4) = —4096, f (5) = 
536,870,912 c) f(2) = 8, f(3) = 176, f(4) =92,672, 
f6) = 25,764, 174, 848 d) f(2) = —5, f@G) = -4, 
fA@ = e f(5) = —32 5. a)Not valid b) f(n) = 
1 — n. Basis step: f(0) = 1 = 1 — 0. Inductive step: if 
f(k) =1-—k, then f(kK+ 1) = f(k) -L=1-k-1l=1 

(K+ 1). o) fm) = 4-—nifn > 0, and f(0) = 2. Basis 
step: f(0) = 2 and f(1) = 3 = 4 — L. Inductive step (with 
k>1): fK4+) = f(k)-1=(4-k)-1=4-(k+ 1). 
d) f(n) = 2'@+D/2) | Basis step: f(0) = 1 = 2bOF)D/2J 
and f(1) = 2 = 2lU+D/2], Inductive step (with k > 1): 
FFD = 2f (kK-1) = 2-214/2) = Q1E/2I41 & Qk tD+D/2) 
e) f(n) = 3”. Basis step: Trivial. Inductive step: For odd 
n, f(n) = 3f(n—1) = 3-3"7! = 3"; and for even 
n> 1, f(n) = 9f(n — 2) = 9-3"? = 3”. 7. There 


are many possible correct answers. We will supply relatively 


simple ones. a)dy41 = a, + 6forn > Landa, = 6 
D) dn41 = a, + 2 forn > landa; = 3 ¢) ayy) = 10a, 
forn > landa,; = 10 dW)apji) = a, forn > 1 and 


aq, =5 9. F(O) =0, F(n) = F(n—1)4+nforn = 1 
11. Pn (0) = 0, Pon + 1) = P(r) +m 13. Let P(n) 


be “fi + fg +--+ + fon-1 = fan.” Basis step: P(1) is 
true because fj) = 1 = fo. Inductive step: Assume that 
P(k) is true. Then fj + fg + ++) + fox-1 + fox+i = 


fox + foesi =  foes2 + froueery. 15. Basis step: 
fofi + fif2 = 0-1 1-l1=P = Fe. Inductive 
step: Assume that fo fi + fif2 +--+ for-ifok = fi. 
Then fofi + fif2 +++ + for-1 for + for forpi + 
fak+i fok+2 fe + for foesr + foeri foes. = 
fox (fox + fort) + fortifor+2 = fox forgot for+i forp2 = 
(fork + for+1) fort2 = oe 17. The number of divisions 
used by the Euclidean algorithm to find gcd( fn+1, fn) is 0 for 
n=0,1forn = 1,andn—1 forn > 2. To prove this result for 
n > 2 we use mathematical induction. For n = 2, one division 
shows that gcd(f3, fo) = ged(2, 1) = gced(1, 0) = 1. Now 
assume that k — | divisions are used to find gced( fx41, fx). 
To find gcd(fx+2, fx+1), first divide fx42 by fx41 to ob- 


tain frxo2 = 1: feat + fe. After one div- ision we 
have gcd(fxi2, fezi) = gced( fey, fe). By the induc- 
tive hypothesis it follows that exactly k — 1 more di- 


visions are required. This shows that k divisions are re- 
quired to find gcd( fx+2, fx+1), finishing the inductive proof. 


19.|A| = -—1. Hence, |A”| = (—1)". It follows that 
fatifn—1 — f2 = (-1)". 21. a) Proof by induction. Ba- 
sis step: Forn = 1, max(—a,;) = -a; = min(q)). 
For n = 2, there are two cases. If a2 > ay, then 
—a, > —d2, so max(—a,, —a2) = —a; = — min(aj, a2). 
If a2 < aj, then —aj < —ay, so max(—aj, —a2) = 
—az = —min(a, az). Inductive step: Assume true for 
k with k > 2. Then max(—a,, —a2,..., —Qx, —dg41) = 
max(max(—d1,..., —@x), —@k41) = max(— min(d],..., ax), 
—ak41) = — min(min(ay,..., ax), ak41) = —min(ay,..., 


ax+1). b) Proof by mathematical induction. Basis step: For 
n = 1, the result is the identity a} + b) = a, + b,. For 
n = 2, first consider the case in which a; + bj > a2 + bo. 
Then max(a; + bj, a2 + bo) = ay + by. Also note 
that aj < max(aqj, a2) and b} <= max(b;, b2), so 
aj + by <<. max(a;, a2) + max(bj, bz). Therefore, 
max(a,}+b 1, d2+b2) =a,+b, < max(aj, a2)-+max(b1, b2). 
The case with aj + bj) < ay + by is similar. In- 
ductive step: Assume that the result is true for k. Then 
max(a, + by, dz + bo, ..., ae + de, ksi + dey) = 
max(max(a; + by), a2 + bo, ..., ae + be), Qey1 + 
bea) < max(max(a1, d2,..., ax) + max(by, bo, ..., by), 
age1 + dey) < max(max(d, a2, ... , dk), 
ae+1) + max(max(dj, bo, ... , de), beyi) = 
max(aj, a2, ..., Ak, k41) + max(by, bo, ..., be, bey). 
c) Same as part (b), but replace every occurrence of “max” by 
“min” and invert each inequality. 23.5 ¢S,andx+yeS 
ifx,y eS. 25.a)0 € S,andifx € S,thenx+2e€S8 
andx —2 € S. b)2 € S,andifx € S,thenx +3 € S. 
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ole S,2¢€S8,3€S,4€S,andifx € S,thenx+5e€58. 
27. a) (0, 1), 1, 1), (2, 1); (0, 2), C1, 2), (2, 2), (G, 2), (4, 2); 
(0, 3), CL, 3), (2, 3), (3, 3), (4, 3), GS, 3), (6, 3); (0, 4), C1, 4), 
(2, 4), (3, 4), (4, 4), (65, 4), (6, 4), (7, 4), (8, 4) b) Let P(n) 
be the statement that a < 2b whenever (a, b) € S is obtained 
by 7 applications of the recursive step. Basis step: P (O) is true, 
because the only element of S obtained with no applications 
of the recursive step is (0, 0), and indeed 0 < 2 - 0. Inductive 
step: Assume that a < 2b whenever (a, b) € S is obtained 
by k or fewer applications of the recursive step, and consider 
an element obtained with k + | applications of the recursive 
step. Because the final application of the recursive step to 
an element (a, b) must be applied to an element obtained 
with fewer applications of the recursive step, we know that 
a < 2b. Add 0 < 2,1 < 2, and 2 < 2, respectively, to obtain 
a<2(b+1),a+1 < 2(b+ 1), anda+2 < 2(b+ 1), as 
desired. c) This holds for the basis step, because 0 < 0. If this 
holds for (a, b), then it also holds for the elements obtained 
from (a, b) in the recursive step, because adding 0 < 2, 1 < 2, 
and 2 < 2, respectively, toa < 2b yieldsa < 2(b + 1), 
a+1 < 2(b+1),anda+2 < 2(b+ 1). 29. a) Define 
S by (1,1) € S, and if (a,b) € S, then (a+ 2,b) € S, 
(a,b +2) € S,and(a+1,b+ 1) € S. All elements put 
in S satisfy the condition, because (1, 1) has an even sum of 
coordinates, and if (a, b) has an even sum of coordinates, then 
so do (a+2, b), (a, b+2), and (a+1, b+ 1). Conversely, we 
show by induction on the sum of the coordinates that if a+b 
is even, then (a, b) € S. If the sum is 2, then (a, b) = (1, 1), 
and the basis step put (a, b) into S. Otherwise the sum is 
at least 4, and at least one of (a — 2, b), (a, b — 2), and 
(a — 1, b — 1) must have positive integer coordinates whose 
sum is an even number smaller than a + b, and therefore must 
be in S. Then one application of the recursive step shows that 
(a,b) € S. b) Define S by (1, 1), C1, 2), and (2, 1) are in S, 
and if (a, b) € S, then (a + 2, b) and (a, b+ 2) are in S. To 
prove that our definition works, we note first that (1, 1), (1, 2), 
and (2, 1) all have an odd coordinate, and if (a, b) has an odd 
coordinate, then so do (a + 2, b) and (a, b + 2). Conversely, 
we show by induction on the sum of the coordinates that if 
(a, b) has at least one odd coordinate, then (a, b) € S. If 
(a,b) = (i, 1) or (a, b) = (1, 2) or (a, b) = (2, 1), then 
the basis step put (a, b) into S$. Otherwise either a or b is 
at least 3, so at least one of (a — 2, b) and (a, b — 2) must 
have positive integer coordinates whose sum is smaller than 
a + b, and therefore must be in S. Then one application of 
the recursive step shows that (a,b) € S. ce) (1,6) € S and 
(2,3) € S, and if (a,b) € S, then (a + 2,b) € S and 
(a,b +6) € S. To prove that our definition works, we note 
first that (1, 6) and (2, 3) satisfy the condition, and if (a, b) 
satisfies the condition, then so do (a + 2, b) and (a, b + 6). 
Conversely we show by induction on the sum of the coordi- 
nates that if (a, b) satisfies the condition, then (a, b) € S. 
For sums 5 and 7, the only points are (1, 6), which the basis 
step put into S$, (2, 3), which the basis step put into S, and 
(4,3) = (2 + 2, 3), which is in S by one application of the 
recursive definition. For a sum greater than 7, either a > 3, or 
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a <2andb = 9, in which case either (a — 2, b) or (a, b — 6) 
must have positive integer coordinates whose sum is smaller 
than a + b and satisfy the condition for being in S. Then 
one application of the recursive step shows that (a, b) € S. 
31. If x is a set or a variable representing a set, then x is a 
well-formed formula. If x and y are well-formed formulae, 
then so are x, (x U y), (xn MN y), and (x — y). 33. a) If 
xéeD = {0, 1, 2,3, 4,5, 6, 7, 8, 9}, then m(x) = x; if 
s§ =tx, where t € D* andx € D, then m(s) = min(m(s), x). 
b) Let t = wx, where w € D* andx € D.Ifw =A, then 
m(st) = m(sx) = min(m(s), x) = min(m(s), m(x)) by 
the recursive step and the basis step of the definition of m. 
Otherwise, m(st) = m((sw)x) = min(m(sw), x) by the 
definition of m. Now m(sw) = min(m/(s), m(w)) by the 
inductive hypothesis of the structural induction, so m(st) = 
min(min(m(s), m(w)), x) = min(m(s), min(m(w), x)) by 
the meaning of min. But min(m(w), x) = m(wx) = m(t) 
by the recursive step of the definition of m. Thus, m(st) = 
min(m(s), m(t)). 35.A® =A and (ux)® = xu¥ forx € Xd, 
u€ d*. 37.w? = dA and w"t! = ww". 39. When the 
string consists of n Os followed by n 1s for some non- negative 
integern 41. Let P(i) be “I(w!) = i-1(w).’ P(O) is true 
because /(w°) = 0 = 0- /(w). Assume P(i) is true. Then 
lw't!) = lw!) = lw) + lw!) = lw) +i - lov) = 
(i +1)-l@w). 43. Basis step: For the full binary tree con- 
sisting of just a root the result is true because n(T) = 1 
and h(T) = 0, and 1 > 2-04 1. Inductive step: As- 
sume that n(7;) > 2h(T,) + 1 and n(T)) > 2h(Th) + 1. 
By the recursive definitions of n(T) and h(T), we have 
n(T) = 1+n(T,)+n(72) andh(T) = 1+max(h(T;), h(1h)). 
Therefore n(T) = 1+ n(Ti) + n(7o) > 14+ 2A(T,) + 
14+ 2h(F)) +1 > 14+2- max(h(T)), h(%))) +2 = 
1+ 2(max(h(7)), A(72)) +1) = 14+ 2A(T). 45. Basis 
step: ang = 0 = O04 O. Inductive step: Assume that 
Gn'.nt = m'’ +n’ whenever (m', n’) is less than (m, n) 
in the lexicographic ordering of N x N. Ifn = O then 
adnan = 4m-ln t1 = m—-14+n+1l=m-+n.lfn > 0, 
then dmn = Gnn-1 +1 = m+n 1l+1=me4+n. 
47. a) Pin.m = Pm because a number exceeding m cannot be 
used in a partition of m. b) Because there is only one way to 
partition 1, namely, 1 = 1, it follows that P;,, = 1. Because 
there is only one way to partition m into 1s, Py.) = 1. When 
n > mit follows that Py,» = Pm.m because a number exceed- 
ing m cannot be used. Pinm = 1+ Pin,m—1 because one extra 
partition, namely, m = m, arises when m is allowed in the par- 
tition. Pinn = Pmn—1+Pm-—n,n ifm > n because a partition of 
m into integers not exceeding n either does not use any ns and 
hence, is counted in P,, ,—1 or else uses an n and a partition of 
m —n, and hence, is counted in Py»—p.n. €) P5 = 7, Po = 11 
49. Let P(n) be “A(n, 2) = 4. Basis step: P(A) is true 
because A(1, 2) = A(O, ACI, 1)) = A(O,2) = 2-2 = 4. 
Inductive step: Assume that P (7) is true, that is, A(n, 2) = 4. 
Then A(n + 1,2) = A(n, Ad + 1, 1)) = A(n, 2) = 4. 
51.a) 16 b) 65,536 53. Use a double induction argument 
to prove the stronger statement: A(m, k) > A(m,/) when 
k > 1. Basis step: When m = 0 the statement is true because 


k > 1 implies that A(O, k) = 2k > 21 = A(0,/). Inductive 
step: Assume that A(m, x) > A(m, y) for all nonnegative 
integers x and y with x > y. We will show that this implies 
that A(m + 1,k) > A(m + 1,1) if k > 1. Basis steps: When 
1=Oandk > 0, A(m+ 1,1) = Oand either A(m+ 1, k) = 2 
or A(m + 1,k) = A(m, Am + 1,k — 1)). If m = 0, 
this is 2A(1,k — 1) = 2 ifm > 0, this is greater than 0 
by the inductive hypothesis. In all cases, A(m + 1, k) > 0, 
and in fact, Adm + 1,k) > 2.1f/ = landk > 1, then 
A(m+1,1) =2 and A(m+1,k) = A(m, A(m+1,k —1)), 
with A(m + 1,k — 1) => 2. Hence, by the inductive hypoth- 
esis, A(m, A(m + 1,k — 1)) => A(m,2) > A(m, 1) = 2. 
Inductive step: Assume that A(m + 1,r) > A(m + 1, 5) for 
alr >s,s=0,1,...,/.Thenifk+1>/+1 it follows that 
A(m+1,k+1) = A(m, A(m+1,k)) > A(m, A(m+1,k)) = 
A(m + 1,/ + 1). 55. From Exercise 54 it follows that 
AG )>AG-) = = AG|) = 2 > fF. 
57. Let P(n) be “F(n) is well-defined.” Then P(0) is true 
because F (0) is specified. Assume that P(k) is true for all 
k <_ n. Then F(n) is well-defined at n because F(n) is 
given in terms of F(0), F(1),..., F( — 1). So P(n) is true 
for all integers n. 59. a) The value of F(1) is ambiguous. 
b) F(2) is not defined because F(0) is not defined. c) F(3) 
is ambiguous and F'(4) is not defined because F () makes 
no sense. d) The definition of F(1) is ambiguous because 
both the second and third clause seem to apply. e) F(2) 
cannot be computed because trying to compute F(2) gives 
FQ)=1+F(F())=14+F(). 6l.a)l1 b)2. o)3 

d)3 e)4 f)4 g)5 63. f9()=[n/a]_ 65. ff) = 
[loglogn] forn > 2, fx (1) =0 


Section 5.4 
1. First, we use the recursive step to write 5! = 5 - 4!. We 
then use the recursive step repeatedly to write 4! = 4 - 3}, 


3! = 3.-2!,2! = 2-1!, and 1! = 1 - O!. Inserting the value 
of 0! = 1, and working back through the steps, we see that 
W=1-1=1,2!=2-1=2-1=2,3!=3-2!=3-2=6, 
446=4-.3!=4-6 = 24,and5! =5-4! = 5-24 = 120. 
3. With this input, the algorithm uses the else clause to find 
that gcd(8, 13) = gcd(13_ mod 8, 8) = gcd(5, 8). It uses this 
clause again to find that gcd(5, 8) = gcd(8 mod 5,5) = 
gcd(3, 5), then to get ged(3, 5) = gced(5 mod 3, 3) = 
gcd(2, 3), then gcd(2, 3) = gced(3 mod 2, 2) = gced(1, 2), 
and once more to get gcd(1, 2) = gcd(2 mod 1,1) = 
gcd(0, 1). Finally, to find gcd(0, 1) it uses the first step 
with a = 0 to find that gcd(O, 1) = 1. Consequently, 


the algorithm finds that gcd(8, 13) = 1. 5. First, be- 
cause n = _ 11 is odd, we use the else clause to see 
that mpower(3, 11,5) =  (mpower (3, 5, 5)? mod 5 - 
3 mod 5) mod 5. We next use the else clause again to see 
that mpower (3, 5,5) = (mpower (3, 2, 5)? mod 5 - 3 
mod 5) mod 5. Then we use the else if clause to see 
that mpower (3, 2,5) = mpower (3, 1, 5)? mod 5. Us- 


ing the else clause again, we have mpower(3, 1,5) = 
(mpower (3, 0, 5)? mod 5 - 3 mod 5) mod 5. Finally, us- 


ing the if clause, we see that mpower(3, 0,5) = 1. 
Working backward it follows that mpower(3, 1,5) = 
(1? mod 5 - 3 mod 5) mod 5 = 3, mpower(3, 2,5) = 


37 mod5 = 4, mpower(3,5,5) = (47 mod5 - 
3 mod 5) mod 5 = _ 3, and finally mpower (3, 11,5) = 
(32 mod 5 - 3 mod 5) mod 5 = 2. We conclude that 
31! mod 5 =2. 


7. procedure mult(n: positive integer, x: integer) 
ifn = | then return x 
else return x + mult(n — 1, x) 
9. procedure sum of odds(n: positive integer) 
ifn = 1 then return | 
else return sum of odds (n — 1)+2n—1 
11. procedure smallest(a,, ..., dy: integers) 
ifn = 1 then return a; 
else return 
min(smallest (a1, ..., An—1); Gn) 
13. procedure modfactorial(n, m: positive integers) 
ifn = | then return 1| 
else return 
(n - modfactorial(n — 1,m)) mod m 
15. procedure gcd(a, b: nonnegative integers) 
{a < b assumed to hold} 
if a = 0 then return b 
else if a = b — a then return a 
else if a < b — a then return gcd(a, b — a) 
else return gcd(b — a, a) 
17. procedure multiply(x, y: nonnegative integers) 
if y = O then return 0 
else if y is even then 
return 2 - multiply (x, y/2) 
else return 2 - multiply (x, (y—1)/2) + x 
19. We use strong induction on a. Basis step: If a = 0, we 
know that gced(0, b) = b for all b > O, and that is precisely 
what the if clause does. Inductive step: Fix k > O, assume 
the inductive hypothesis—that the algorithm works correctly 
for all values of its first argument less than k—and consider 
what happens with input (k, b), where k < b. Because k > 0, 
the else clause is executed, and the answer is whatever the 
algorithm gives as output for inputs (b mod k, k). Because 
b mod k < k, the input pair is valid. By our inductive hy- 
pothesis, this output is in fact ged(b mod k, k), which equals 
gcd(k, b) by Lemma | in Section 4.3. 21. Ifm = 1, then 
nx = x, and the algorithm correctly returns x. Assume that 
the algorithm correctly computes kx. To compute (k + 1)x it 
recursively computes the product of k + 1— 1 = k and x, 
and then adds x. By the inductive hypothesis, it computes that 
product correctly, so the answer returned is kx +x = (k+1)x, 
which is correct. 
23. procedure square(n: nonnegative integer) 
ifn = 0 then return 0 
else return square (n —1)+2(n—1)+1 
Let P(n) be the statement that this algorithm correctly com- 
putes n*. Because 0? = 0, the algorithm works correctly 
(using the if clause) if the input is 0. Assume that the algo- 
rithm works correctly for input k. Then for input k + 1, it 
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gives as output (because of the else clause) its output when 
the input is k, plus 2(k + 1 — 1) + 1. By the inductive 
hypothesis, its output at k is kK, so its output at k + 1 is 
P4%k+1—-14+1=k?4+2k4+1=(k +1)’, as desired. 
25.n multiplications versus 2”. 27. O(log n) versus n 
29. procedure a(n: nonnegative integer) 

if n = 0 then return | 

else ifn = | then return 2 

else return a(n — 1) - a(n — 2) 
31. Iterative 
33. procedure iferative(n: nonnegative integer) 

ifn = 0 then z := 1 

else ifn = | then z := 2 


else 
Pao — a | 
y= 
zi=3 
fori := 1ton—2 
wi=x+y+zZ 
xi=y 
yrs 
Zi=w 


return z {z is the nth term of the sequence } 


35. We first give a recursive procedure and then an iterative 
procedure. 

procedure r(n: nonnegative integer) 

ifn < 3 then return 2n + 1 

else return r(n — 1) - (r(n — 2))? - (r(n — 3))3 


procedure i(n: nonnegative integer) 
ifn = 0 then z := 1 
else ifn = | then z := 3 


else 

Petal 
y2=3 
CoS s5 


fori := 1ton—2 
wisz-y?-x3 


xi=y 
Yas 
Zi=w 


return z {z is the nth term of the sequence} 
The iterative version is more efficient. 


37. procedure reverse(w: bit string) 

n := length(w) 

ifn < | then return w 

else return 

substr(w, n, n)reverse (substr (w, 1,n — 1)) 
{substr(w, a, b) is the substring of w consisting of 
the symbols in the ath through bth positions} 

39. The procedure correctly gives the reversal of A as (basis 
step), and because the reversal of a string consists of its last 
character followed by the reversal of its first n — 1 charac- 
ters (see Exercise 35 in Section 5.3), the algorithm behaves 
correctly when n > 0 by the inductive hypothesis. 41. The 
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algorithm implements the idea of Example 14 in Section 5.1. 
Ifn = 1 (basis step), place the one right triomino so that its 
armpit corresponds to the hole in the 2 x 2 board. Ifn > 1, then 
divide the board into four boards, each of size 2"~! x 2"7!, 
notice which quarter the hole occurs in, position one right tri- 
omino at the center of the board with its armpit in the quarter 
where the missing square is (see Figure 7 in Section 5.1), and 
invoke the algorithm recursively four times—once on each of 
the 2”—! x 2”—! boards, each of which has one square missing 
(either because it was missing to begin with, or because it is 
covered by the central triomino). 


43. procedure A(m, n: nonnegative integers) 
if m = O then return 2n 
else if n = 0 then return 0 
else if n = 1 then return 2 
else return A(m — 1, A(m,n — 1)) 
45. bdafghzpok 


bdafg hzpok 


-\ & & 
ee oT 
a las 
4 \/ 

‘ \ 


hpz 


A via 


abdfg hkopz 


abdfghkopz 


47. Let the two lists be 1, 2,...,m — 1,m +n — 1 and 
m,m+1,...,m-+n — 2,m +n, respectively. 49. If 
n = 1, then the algorithm does nothing, which is correct be- 
cause a list with one element is already sorted. Assume that 
the algorithm works correctly form = 1 throughn = k. If 
n = k-+1, then the list is split into two lists, L; and L2. By the 
inductive hypothesis, mergesort correctly sorts each of these 
sublists; furthermore, merge correctly merges two sorted lists 
into one because with each comparison the smallest element 
in L; UL? not yet put into Lis putthere. 51.O(n) 53.6 
55. O(n”) 


Section 5.5 


1. Suppose that x = 0. The program segment first assigns the 
value | to y and then assigns the value x + y=0+1=1to 
z. 3. Suppose that y = 3. The program segment assigns the 
value 2 to x and then assigns the value x + y = 2+3 = 5 toz. 


Because y = 3 > Oit then assigns the valuez+1=5+1=6 
to z. 
5. (pa condition1){S\}q 

(pA a=condition| A condition2){S2}q 


(pA condition! A —condition2 
++ A acondition(n — 1){Sy}q 
‘. p{if condition\ then S,; 
else if condition2 then S2;...; else S,}q 
7. We will show that p : “power = x'~! andi <n+1” isa 
loop invariant. Note that p is true initially, because before the 
loop starts, i = 1 and power = 1 = x9 = x1-1, Next, we must 
show that if p is true andi < n after an execution of the loop, 
then p remains true after one more execution. The loop incre- 
ments i by 1. Hence, because i < n before this pass, i <n+1 
after this pass. Also the loop assigns power - x to power. By 
the inductive hypothesis we see that power is assigned the 
value x'—! . x = x'. Hence, p remains true. Furthermore, the 
loop terminates after n traversals of the loop withi = n+ 1 
because i is assigned the value | prior to entering the loop, is 
incremented by 1 on each pass, and the loop terminates when 
i > n. Consequently, at termination power = x”, as desired. 
9. Suppose that p is “m and n are integers.” Then if the con- 
dition n < Ois true, a = —n = |n| after S; is executed. If the 
conditionn < Ois false, then a =n = |n| after S) is executed. 
Hence, p{S}q is true where g is p A (a = |n|). Because S 
assigns the value 0 to both k and x, it is clear that g{S2}r is 
true where r is g A (k = 0) A (x = 0). Suppose that r is true. 
Let P(k) be “x = mk and k < a.” We can show that P(k) isa 
loop invariant for the loop in $3. P(O) is true because before 
the loop is entered x = 0 = m-Oand0 < a. Now assume P(k) 
is true andk < a. Then P(k + 1) is true because x is assigned 
the value x +m = mk +m = m(k + 1). The loop terminates 
when k = a, and at that point x = ma. Hence, r{S3}s is true 
where s is “a = |n| and x = ma.’ Now assume that s is 
true. Then ifn < 0 it follows thata = —n,so x = —mn. 
In this case $4 assigns —x = mn to product. Ifn > O then 
x = ma = mn, so Sq assigns mn to product. Hence, s{S4}t 
istrue. 11. Suppose that the initial assertion p is true. Then 
because p{S}qo is true, go is true after the segment S is exe- 
cuted. Because go — q is true, it also follows that q; is true 
after S is executed. Hence, p{S}qj is true. 13. We will use 
the proposition p, “gcd(a, b) = gcd(x, y) and y > 0,” as the 
loop invariant. Note that p is true before the loop is entered, 
because at that point x = a, y = b, and y is a positive inte- 
ger, using the initial assertion. Now assume that p is true and 
y > 0; then the loop will be executed again. Inside the loop, x 
and y are replaced by y and x mod y, respectively. By Lemma 
1 of Section 4.3, gcd(x, y) = gced(y, x mod y). Therefore, 
after execution of the loop, the value of gcd(x, y) is the same 
as it was before. Moreover, because y is the remainder, it is 
at least 0. Hence, p remains true, so it is a loop invariant. 
Furthermore, if the loop terminates, then y = 0. In this case, 
we have gcd(x, y) = x, the final assertion. Therefore, the 
program, which gives x as its output, has correctly computed 


gcd(a, b). Finally, we can prove the loop must terminate, be- 
cause each iteration causes the value of y to decrease by at 
least 1. Therefore, the loop can be iterated at most b times. 
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1. Let P(n) be the statement that this equation holds. Basis 
step: P(1) says 2/3 = 1 — (1/3!), which is true. Inductive 
step: Assume that P(k) is true. Then 2/3 + 2/9 + 2/27 + 
ve $ 2/37 +2/3"t! = 1 — 1/3" + 2/3"+! (by the inductive 
hypothesis), and this equals 1 — 1/3"*!, as desired. 3. Let 
P(n) be “1-1+2-2+4+---4+n-277! = (n—1)27 41” 
Basis step: P(1) is true because 1-1 = 1 = (1 — 1)2! +1. 
Inductive step: Assume that P(k) is true. Then 1- 1+ 2-2+ 
vee th D1 4 (e41)-2% = (k— 12% 4144+ 12" = 
2k- 2k +1 =[(e + 1) — 172**1 +1. 5. Let P(n) be 
“1/1 - 4) +--+ + 1/[Bn — 2)Bn + 1D] = n/GBn + 1)” 
Basis step: P(1) is true because 1/(1 - 4) = 1/4. Inductive 
step: Assume P(k) is true. Then 1/(1 - 4) +--+ + 1/[Gk — 
23k + )]+1/[Bk+ )Bk+4)] = k/Bk+1) +1/[Bk 4 
Gk + 4) = [KGk +4) + H/[1Bk + DGBE+ 4) = 
[3k + I)(k + DI/(Bk + DBE+4)] = K+1/Bk +4). 
7. Let P(n) be “2” > n>.” Basis step: P(10) is true because 
1024 > 1000. Inductive step: Assume P(k) is true. Then 
(k+13 =243h243k4+1<349P2 <P4+Rh = 
2k? < 2-.2* = 24+! 9. Let P(n) be “a — b is a factor 
of a” — b"”” Basis step: P(1) is trivially true. Assume P(k) 
is true. Then a*+! — pk+! — gkt! — gpk + abk — pet! — 
a(ak — bk) + bk(a — b). Then because a — b is a factor of 
ak — bk anda — bisa factor of a — b, it follows that a — b 
is a factor of att! — b‘t+!, 11. Basis step: Whenn = 1, 
6't! 4 7"! — 3647 = 43. Inductive step: Assume the 
inductive hypothesis, that 43 divides 6"*! + 72"—!; we must 
show that 43 divides 6+? + 7?"+!. We have 6+? 4 72741 — 
6- ort! +49. 7n-l =6. ort! +6: q2n-l 43 qan-l — 
6(6"*! + 7°"-!) + 43. 7°"-!. By the inductive hypothesis 
the first term is divisible by 43, and the second term is clearly 
divisible by 43; therefore the sum is divisible by 43. 13. Let 
P(n) be “a+(a+d)+---+(a+nd) = (n+1)(a+nd)/2.” 
Basis step: P(1) is true because a + (a+ d) = 2a+d = 
2(2a + d)/2. Inductive step: Assume that P(k) is true. Then 
a+(a+d)+---+ (a+ kd) + [a+ (K+ 1)d] = 
(k + 1)(Qa+kd)/2+a+ (k+1)d = 5(2ak +2a+k*d 4 
kd + 2a + 2kd + 2d) = 5(2ak + 4a + k?d + 3kd + 2d) = 
5(k + 2)[2a+ (k + 1)d]. 15. Basis step: This is true for 
n = 1 because 5/6 = 10/12. Inductive step: Assume that 


the equation holds for n = k, and consider n = k + 1. Then 
yee i+4 _ ye i+4 : k+5 _ 
i=l i@+DG4+2) ~ i=l iG4+D042) * &+DE4+2)K43) 

i 


k(3k+7) 45 : ‘ : 
WEED ED + KEDELDERS (by the inductive hypothesis) 
= 1 5 (ROR ne Bay 1 . 
~ FD) 2 T k+3 ZF DEEDES) 
[kK(3k +7) (k + 3) + 2(k+5)] TED ELDOD 


3) 2 4 | 1 | 
(3k3+16k? + 23k + 10) = sernanery ¢ Gkt+1N)K4 


2 1 ' ; (k+1)(3(kK+1)+7) 
DD” = sappy © GA+ 104+) = ap EaERD HD? 88 


desired. 17. Basis step: The statement is true forn = 1 be- 


Answers to Odd-Numbered Exercises S-39 


cause the derivative of g(x) = xe* is x -e* +e* = (x+ le* 
by the product rule. Inductive step: Assume that the state- 
ment is true forn = k,i.e., the kth derivative is given by 
g = (x +k)e*. Differentiating by the product rule gives the 
(k+1)st derivative: g4+) = (x+ke* +e* = [x+(k+ De’, 
as desired. 19. We will use strong induction to show that f;, 
is even ifn = 0 (mod 3) and is odd otherwise. Basis step: This 
follows because fp = 0 is even and f| = 1 is odd. Inductive 
step: Assume that if j < k, then f; is even if 7 = 0 (mod 3) 
and is odd otherwise. Now suppose k + 1 = 0 (mod 3). Then 
Ski = Se + fe-1 is even because fy and f,—1 are both odd. 
Ifk + 1 = 1 (mod 3), then fx41 = fx + fe—1 18 odd because 
fx is even and fx—1 1s odd. Finally, if k + 1 = 2 (mod 3), then 
Se+i = fe + fe-1 is odd because f; is odd and f,_1 is even. 
21. Let P(n) be the statement that f, fn+ frei fog = Sntk+l 
for every nonnegative integer k. Basis step: This consists of 
showing that P(0) and P(1) both hold. P(Q) is true because 
Sk fo + fest fi = fer ->O+ fest: 1 = fi. Because 
Sk fit ferif2 = fe + frri = fk+2, it follows that P(1) 
is true. Inductive step: Now assume that P(j) holds. Then, 
by the inductive hypothesis and the recursive definition of the 
Fibonacci numbers, it follows that fr+1 fj+1 + fk+2fj+2 = 
Se (fj-1 + SL) + Seri t+ fi4 = Se fia + Seti fj) + 
(fk fj t+ fer fiz = fj-t4etit fitezi = fj+4+2- This 
shows that P(j + 1) istrue. 23. Let P(n) be the statement 
+i +++ +02 = Inlngi + 2. Basis step: P(0) and P(1) 
both hold because JF = 27 = 2-142 = Il; + 2 and 
B+h = 241 = 1-342 = his + 2. Inductive 
step: Assume that P(k) holds. Then by the inductive hypoth- 
esis 3 +17 +--+ PR +02, = leo +24+3,, = 
Inet + Ieei) +2 = Ikrilkt2 + 2. This shows that 
P(k + 1) holds. 25. Let P(n) be the statement that the 
identity holds for the integer n. Basis step: P(1) is obvi- 
ously true. Inductive step: Assume that P(k) is true. Then 
cos((k+1)x)+i sin((k+1)x) = cos(kx+x)+i sin(kx +x) = 
coskx cosx — sinkx sinx + i(sinkx cosx + coskx sin x) = 
cos x(cos kx + isinkx)(cosx + isinx) = (cosx + 
isinx)‘(cosx +isinx) = (cos x + i sin x)erl, It fol- 
lows that P(kK + 1) is true. 27. Rewrite the right-hand side 
as 2+! (n2 —2n +3) —6. Forn = | we have 2 = 4-2—6. As- 
sume that the equation holds forn = k, and considern = k+1. 
Then et 7224 = rh, 722) + e+ 122k = 21 G2 
2k +3) — 64 (k? + 2k + 1)2+! (by the inductive hypo- 
thesis) = 2*+1(2k? + 4) -6 = 247k? + 2) -6 = 
220k + 1)? — 2k +1) +3] —6. 29. Let P(n) be the 
statement that this equation holds. Basis step: In P(2) both 
sides reduce to 1/3. Inductive step: Assume that P (k) is true. 
Then 244) 1/G?—-1) = (Hh 1/0? - D) + 11 
1)? — 1) = &-— DGBE+2)/(4kR+ D+ 1/1K+ 1)? - 1) 
by the inductive hypothesis. This simplifies to (k — 1)(3k 4 
2) /[4k(k + LD) + 1/(k? + 2k) = (3k3 + 5k7)/T4k(k + Ik + 
2)) = (A+ 1) — 1[3(A+ 1) +2)}/[4(k+ 1) (k+2)], which is 
exactly what P(k+1) asserts. 31. Let P(n) be the assertion 
that at least n + 1 lines are needed to cover the lattice points in 
the given triangular region. Basis step: P(Q) is true, because 
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we need at least one line to cover the one point at (0, 0). Induc- 
tive step: Assume the inductive hypothesis, that at least k + 1 
lines are needed to cover the lattice points with x > 0, y > 0, 
and x + y < k. Consider the triangle of lattice points defined 
by x > 0, y > 0,andx+y <k-+1. By way of contradiction, 
assume that k + 1 lines could cover this set. Then these lines 
must cover the k + 2 points on the line x + y = k +1. 
But only the line x + y = k + 1 itself can cover more than 
one of these points, because two distinct lines intersect in at 
most one point. Therefore none of the & + 1 lines that are 
needed (by the inductive hypothesis) to cover the set of lattice 
points within the triangle but not on this line can cover more 
than one of the points on this line, and this leaves at least one 
point uncovered. Therefore our assumption that k + 1 lines 
could cover the larger set is wrong, and our proof is complete. 
33. Let P(n) be BE = MA‘M7!. Basis step: Part of the given 
conditions. Inductive step: Assume the inductive hypothesis. 
Then B‘+! = BBK = MAM~'BK = MAM7~!MA‘M7! (by 
the inductive hypothesis) = MAIA‘M~! = MAA‘M7! = 
MA‘t!M-!, 35. We prove by mathematical induction the 
following stronger statement: For every n > 3, we can write 
n! as the sum of n of its distinct positive divisors, one of which 
is 1. That is, we can write n! = aj +a2+---+a,, where each 
a; is a divisor of n!, the divisors are listed in strictly decreasing 
order, and a; = 1. Basis step: 3! = 3 +2 + 1. Inductive step: 
Assume that we can write k! as a sum of the desired form, say 
k! = aj + ao +--+ + ag, where each aq; is a divisor of n!, 
the divisors are listed in strictly decreasing order, anda, = 1. 
Consider (k + 1)!. Then we have (k + 1)! = (K+ Ik! = 
(K+ 1)(ay +42 +--+ + ax) = (K+ May + (K+ lag + +--+ 
(A+ Vag = (K+ Var +(k+ Ian +--++k- ay + ag. Be- 
cause each a; was a divisor of k!, each (k + 1)a; is a divisor of 
(k+1)!. Furthermore, k- ax = k, whichis a divisor of (k+1)!, 
and ax = 1, so the new last summand is again 1. (Notice also 
that our list of summands is still in strictly decreasing order.) 
Thus we have written (k+ 1)! inthe desired form. 37. When 
n = | the statement is vacuously true. Assume that the state- 
ment is true for n = k, and consider k + | people standing 
in a line, with a woman first and a man last. If the kth person 
is a woman, then we have that woman standing in front of the 
man at the end. If the kth person is a man, then the first k peo- 
ple in line satisfy the conditions of the inductive hypothesis 
for the first k people in line, so again we can conclude that 
there is a woman directly in front of a man somewhere in the 
line. 39. Basis step: When n = | there is one circle, and 
we can color the inside blue and the outside red to satisfy the 
conditions. Inductive step: Assume the inductive hypothesis 
that if there are k circles, then the regions can be 2-colored 
such that no regions with a common boundary have the same 
color, and consider a situation with k + | circles. Remove one 
of the circles, producing a picture with k circles, and invoke 
the inductive hypothesis to color it in the prescribed manner. 
Then replace the removed circle and change the color of ev- 
ery region inside this circle. The resulting figure satisfies the 
condition, because if two regions have a common boundary, 
then either that boundary involved the new circle, in which 


case the regions on either side used to be the same region and 
now the inside portion is different from the outside, or else 
the boundary did not involve the new circle, in which case 
the regions are colored differently because they were colored 
differently before the new circle was restored. 41.Ifn = 1 
then the equation reads 1 - 1 = 1 - 2/2, which is true. Assume 
that the equation is true for n and consider it for n + 1. Then 


Wes = 1 (iS t) = D@i- ) (Dit t) + 


bety=i- 4 = yor p(t Ee it)+ 
a (4 Pi@i-)) aa (a 27 —1) 
> = t) + m= (GH -n?) a 
(by the inductive hypothesis) = Beem Cini) 

ot nt” GEVG+) 43, Let T(x) be the state- 


ment that the sequence of towers of 2 is eventually constant 
modulo n. We use strong induction to prove that 7 (n) is true 
for all positive integers n. Basis step: Whenn = 1 (andn = 2), 
the sequence of towers of 2 modulo n is the sequence of all Os. 
Inductive step: Suppose that k is an integer with k > 2. Sup- 
pose that T(j) is true for 1 < 7 < k — 1. In the proof of the 
inductive step we denote the rth term of the sequence mod- 
ulo n by a,. First suppose k is even. Let k = 2°g where s > 1 
and q < k is odd. When j is large enough, aj_2 = s, and for 
such j,aj = 22? isa multiple of 2°. It follows that for suffi- 
ciently large j, a; = 0 (mod 2°). Hence, for large enough i, 2° 
divides aj+1 — aj. By the inductive hypothesis T (q) is true, so 
the sequence a1, a2, a3, ... is eventually constant modulo q. 
This implies that for large enough i, g divides aj; — a;. Be- 
cause gcd(qg, 2°) = 1 and for sufficiently large i both g and 
2° divide aj+1 — a;, k = 2°q divides a;+ — a; for sufficiently 
large i. Hence, for sufficiently large i, aj; — a; = 0 (mod k). 
This means that the sequence is eventually constant modulo k. 
Finally, suppose k is odd. Then gcd(2, k) = 1, so by Euler’s 
theorem (found in elementary number theory books, such as 
[Ro10]), we know that 2?) = 1 (mod k). Let r = $(k). Be- 
cause r < k, by the inductive hypothesis T(r), the sequence 
a\, a2, 43, ... is eventually constant modulo r, say equal to c. 
Hence for large enough 7, for some integer t;, aj = tir +c. 
Hence aj+1 24 gurte (2")"2° = 2° (mod k). 
This shows that aj, a2, ... is eventually constant modulo k. 
45.a)92 b)91 c)91 d)91 e)91 f)91 47. The 
basis step is incorrect because n 4 1 for the sum shown. 
49. Let P(n) be “the plane is divided into n? —n+2 regions by 
n circles if every two of these circles have two common points 
but no three have a common point.” Basis step: P(1) is true 
because a circle divides the plane into 2 = 17 — 1 +2 regions. 
Inductive step: Assume that P(k) is true, that is, k circles 
with the specified properties divide the plane into k? — k + 2 
regions. Suppose that a (k + 1)st circle is added. This cir- 
cle intersects each of the other k circles in two points, so 
these points of intersection form 2k new arcs, each of which 
splits an old region. Hence, there are 2k regions split, which 
shows that there are 2k more regions than there were pre- 
viously. Hence, k + 1 circles satisfying the specified prop- 


erties divide the plane into k27 — k + 2+ 2k = (k?+ 
2%+1—-k+tN)4+2 = k&4+ 1? — (4+ 1) 4+ 2 re 
gions. 51. Suppose V2 were rational. Then /2 = a/b, 
where a and Db are positive integers. It follows that the set 
S = {nJ/2 | n € N}NNisa nonempty set of positive integers, 
because b./2 =a belongs to S. Let t be the least element of S, 
which exists by the well-ordering property. Then ¢ = s V2 for 
some integer s. We have t —s = sJ2—-s= s(/2 1),sot—s 
is a positive integer because 2 > 1. Hence, t—s belongs to S. 
This is acontradiction because t—s = s./2—s <5. Hence, af, 
is irrational. 53. a) Let d = gcd(aj, a2, ..., a,). Thend is 
a divisor of each a; and so must be a divisor of gcd(dy_1, ay). 
Hence, d is a common divisor of a), a2, ... , Gn—2, and 
gcd(an—1, dn). To show that it is the greatest common divisor 
of these numbers, suppose that c is acommon divisor of them. 
Then c is a divisor of a; fori = 1,2,...,n — 2 and a divisor 
of gcd(an—1, dy), SO it is a divisor of d,—1 and a,. Hence, c 
is acommon divisor of a), d2,..., Gn—1, and a,. Hence, it is 
a divisor of d, the greatest common divisor of a1, d2,..., dy. 
It follows that d is the greatest common divisor, as claimed. 
b) If = 2, apply the Euclidean algorithm. Otherwise, ap- 
ply the Euclidean algorithm to a,_; and a,, obtaining d = 
gcd(a,—1, d,), and then apply the algorithm recursively to a1, 
a2,...,4n-2,4d. 55. f(n) = n2. Let P(n) be “f(n) = n?.” 
Basis step: P (1) is true because f (1) = 1 = 12, which follows 
from the definition of f. Inductive step: Assume f (n) = n°. 
Then fn +1) = f(atl]) —1)4+2@4+)-l=fmt+ 
Qnt+1 =n? 4+2n4+1= (+1). 57.4), 0, 1, 00, 
01, 11, 000, 001, 011, 111, 0000, 0001, 0011, O111, 1111, 
00000, 00001, 00011, 00111, 01111, 11111 b)S = {afb | a 
is a string of m Os and £ is a string of n 1s, m > 0,n > O} 
59. Apply the first recursive step to A to get () € B. Apply 
the second recursive step to this string to get ()() € B. Ap- 
ply the first recursive step to this string to get (OQ) € B. 
By Exercise 62, (())) is not in B because the number of left 
parentheses does not equal the number of right parentheses. 
61.4,0,(0),OQ 63.a)0 b)—2 c)2 dO 


65. 
procedure generate(n: nonnegative integer) 
if n is odd then 
S := S(n — 1) {the S constructed by generate(n — 1)} 
T := T(n — 1) {the T constructed by generate(n — 1)} 
else if n = 0 then 
S:=@ 
T := {a} 
else 
S’ := S(n — 2) {the S constructed by generate(n — 2)} 
T’ := T(n — 2) {the T constructed by generate(n — 2)} 
T :=T'U{(x)|x € T’US’ A length(x) =n — 2} 
S:= S’U{xy|x €T’AyeT’'US' A length(xy) = n} 
{T U S is the set of balanced strings of length at most n} 
67. If x < y initially, then x := y is not executed, sox < y 
is a true final assertion. If x > y initially, then x := y is 
executed, so x < y is again a true final assertion. 
69. procedure zerocount(a,, a2, ..., An: list of integers) 
ifn = 1 then 
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if a; = O then return 1 
else return 0 


else 
if a, = 0 then return zerocount (a), a2, ...,4n—1) +1 
else return zerocount (aj, a2, ..., An—1) 


71. We will prove that a(m) is a natural number and a(n) <n. 
This is true for the base case n = 0 because a(0) = 0. Now 
assume that a(n — 1) is anatural number and a(n—1) <n—1. 
Then a(a(n — 1)) is a applied to a natural number less than or 
equal to n — 1. Hence, a(a(n — 1)) is also a natural number 
minus than or equal to n — 1. Therefore, n — a(a(n — 1)) isn 
minus some natural number less than or equal to n—1, whichis 
a natural number less than or equal ton. 73. From Exercise 
72, a(n) = |(n+1)u| anda(n— 1) = [np]. Because pv < 1, 
these two values are equal or they differ by 1. First suppose that 
pn—|pun| < 1—p.Thisis equivalent to w(n+1) < 1+ [un]. 
If this is true, then |w(m + 1)| = [wn]. On the other hand, 
if un — |un| > 1— yp, then wn+ 1) => 1+ [pn], 
so |u(n + 1)] = Lun] + 1, as desired. 75. f(0) = 1, 
m(O) = 0; f0) = 1,m() = 0; fQ) = 2,mQ) = 1; 
£3) = 2,m@3) = 2; f(4) = 3,m(4) = 2; f(S) = 3, 
m(5) = 3; f(6) 4, m(6) 4, f/@7) = 5,m(7) = 4; 
f(8) = 5,m(8) = 5; f(9) = 6,m(9) = 6 77. The last 
occurrence of 7 is in the position for which the total number of 
1s, 2s, ..., ns all together is that position number. But because 
ay is the number of occurrences of k, this is just Wiel ay, as 
desired. Because f(n) is the sum of the first n terms of the 
sequence, f(f(m)) is the sum of the first f() terms of the 
sequence. But because f (7) is the last term whose value is n, 
this means that the sum is the sum of all terms of the sequence 
whose value is at most n. Because there are a, terms of the 
sequence whose value is k, this sum is ea k- ag, as desired 


CHAPTER 6 


Section 6.1 


1. a)5850 ~—-b) 343s 3. a) 4!9— by 5! 5.427.268 
9.676 11.28 13.n + 1 (counting the empty string) 
15. 475,255 (counting the empty string) 17. 1,321,368,961 
19.a)729 b)256 c)1024 d)64 21.a) Seven: 56, 
63, 70, 77, 84, 91, 98  b) Five: 55, 66, 77, 88, 99 
c)One: 77 23.a)128 b)450 ©)9 d)675 _ e) 450 
f) 450 g)225 h)75 25.a)990 b)500 c)27 27.3°° 
29. 52,457,600 31. 20,077,200 33. a) 37,822,859,361 

b) 8,204,716,800 c) 40,159,050, 880 — d) 12,113,640,000 
e) 171,004,205,215  f) 72,043,541,640 —_ g) 6,230,721,635 
h) 223,149,655 35. a) 0 b) 120 e) 720 d) 2520 37.a)2 
ifn = 1,2ifn = 2,0ifn > 3 b)2”? forn > 1; 1if 
n=1¢)2m—1) 39.(n +1)” 41. If nis even, 2"/?; 
if n is odd, 2"+)/2 43.a) 175 b) 248 c¢) 232 d) 84 
45.60 47.a)240 b)480 c)360 49.352 51.147 
53.33 55. a) 9,920,671,339,261,325,541,376 ~ 9.9 x 
107! b) 6,641, 514,961,387,068,437,760 ~ 6.6 x 
10?! ¢) About 314,000 years 57. 54(6499976 — 1)/63 


S-42 Answers to Odd-Numbered Exercises 


59. 7,104,000,000,000 61.1619 +4 16° + 168 
63. 666,667 65.18 67.17 69.22 71.Let P(m) be 
the sum rule for m tasks. For the basis case take m = 2. This 
is just the sum rule for two tasks. Now assume that P(m) is 
true. Consider m + | tasks, 71, To, ...,Tim, Tm+1, which can 
be done in, 2, ..., 1m, m+1 Ways, respectively, such that 
no two of these tasks can be done at the same time. To do one 
of these tasks, we can either do one of the first m of these or 
do task T,,41. By the sum rule for two tasks, the number of 
ways to do this is the sum of the number of ways to do one of 
the first m tasks, plus n,,41. By the inductive hypothesis, this 
isny +no+---+Nm +Nm41, as desired. 73. n(n — 3)/2 


Section 6.2 


1. Because there are six classes, but only five weekdays, the 
pigeonhole principle shows that at least two classes must be 
held on the same day. 3.a)3 b) 14 5. Because there are 
four possible remainders when an integer is divided by 4, 
the pigeonhole principle implies that given five integers, at 
least two have the same remainder. 7.Leta,a+1,..., 
a +n — 1 be the integers in the sequence. The integers 
(a+i)modn,i = 0,1,2,...,n — 1, are distinct, because 
0 < (a+ j) —(a+k) <n whenever 0 <k < j <n—1. Be- 
cause there are n possible values for (a + i) mod n and there 
are n different integers in the set, each of these values is taken 
on exactly once. It follows that there is exactly one integer in 
the sequence that is divisible byn. 9.4951 11. The mid- 
point of the segment joining the points (a, b, c) and (d, e, f) 
is ((a+d)/2, (b+e)/2, (c+ f)/2). It has integer coefficients 
if and only if a and d have the same parity, b and e have the 
same parity, and c and f have the same parity. Because there 
are eight possible triples of parity [such as (even, odd, even)], 
by the pigeonhole principle at least two of the nine points have 
the same triple of parities. The midpoint of the segment join- 
ing two such points has integer coefficients. 13. a) Group 
the first eight positive integers into four subsets of two inte- 
gers each so that the integers of each subset add up to 9: {1, 8}, 
{2, 7}, {3, 6}, and {4, 5}. If five integers are selected from the 
first eight positive integers, by the pigeonhole principle at least 
two of them come from the same subset. Two such integers 
have a sum of 9, as desired. b) No. Take {1, 2, 3, 4}, for exam- 
ple. 15.4 17.21,251 19. a) If there were fewer than 9 
freshmen, fewer than 9 sophomores, and fewer than 9 juniors 
in the class, there would be no more than 8 with each of these 
three class standings, for a total of at most 24 students, con- 
tradicting the fact that there are 25 students in the class. b) If 
there were fewer than 3 freshmen, fewer than 19 sophomores, 
and fewer than 5 juniors, then there would be at most 2 fresh- 
men, at most 18 sophomores, and at most 4 juniors, for a total 
of at most 24 students. This contradicts the fact that there are 
25 students in the class. 21.4, 3, 2, 1, 8, 7, 6, 5, 12, 11, 
10, 9, 16, 15, 14,13 23. Number the seats around the table 
from | to 50, and think of seat 50 as being adjacent to seat 1. 
There are 25 seats with odd numbers and 25 seats with even 
numbers. If no more than 12 boys occupied the odd-numbered 


seats, then at least 13 boys would occupy the even-numbered 
seats, and vice versa. Without loss of generality, assume that 
at least 13 boys occupy the 25 odd-numbered seats. Then at 
least two of those boys must be in consecutive odd-numbered 
seats, and the person sitting between them will have boys as 
both of his or her neighbors. 
25. procedure long(a,, ..., dy: positive integers) 

{first find longest increasing subsequence} 

max := 0; set := 00... 00 {n bits} 


for i := 1 to 2” 
last := 0; count := 0, OK := true 
for j := 1 ton 


if set(j) = 1 then 
if a; > last then last := a; 
count := count + | 
else OK := false 
if count > max then 
max := count 
best := set 
set := set+ | (binary addition) 
{max is length and best indicates the sequence} 
{repeat for decreasing subsequence with only 

changes being a; < last instead of a; > last 

and last := oo instead of last := 0} 
27. By symmetry we need prove only the first statement. Let 
A be one of the people. Either A has at least four friends, or A 
has at least six enemies among the other nine people (because 
3+ 5 < 9). Suppose, in the first case, that B, C, D, and E 
are all A’s friends. If any two of these are friends with each 
other, then we have found three mutual friends. Otherwise 
{B, C, D, E} is a set of four mutual enemies. In the second 
case, let {B, C, D, E, F, G} be a set of enemies of A. By 
Example 11, among B, C, D, E, F, and G there are either 
three mutual friends or three mutual enemies, who form, with 
A, a set of four mutual enemies. 29. We need to show two 
things: that if we have a group of n people, then among them 
we must find either a pair of friends or a subset of n of them 
all of whom are mutual enemies; and that there exists a group 
of n — | people for which this is not possible. For the first 
statement, if there is any pair of friends, then the condition is 
satisfied, and if not, then every pair of people are enemies, so 
the second condition is satisfied. For the second statement, if 
we have a group of n — 1 people all of whom are enemies of 
each other, then there is neither a pair of friends nor a subset 
of n of them all of whom are mutual enemies. 31. There 
are 6,432,816 possibilities for the three initials and a birthday. 
So, by the generalized pigeonhole principle, there are at least 
[37,000,000/6,432,816] = 6 people who share the same 
initials and birthday. 33. Because 800,001 > 200,000, the 
pigeonhole principle guarantees that there are at least two 
Parisians with the same number of hairs on their heads. The 
generalized pigeonhole principle guarantees that there are at 
least [800,001/200,000] = 5 Parisians with the same num- 
ber of hairs on their heads. 35.18 37. Because there are 
six computers, the number of other computers a computer is 
connected to is an integer between 0 and 5, inclusive. How- 
ever, 0 and 5 cannot both occur. To see this, note that if some 


computer is connected to no others, then no computer is con- 
nected to all five others, and if some computer is connected 
to all five others, then no computer is connected to no others. 
Hence, by the pigeonhole principle, because there are at most 
five possibilities for the number of computers a computer is 
connected to, there are at least two computers in the set of six 
connected to the same number of others. 39. Label the com- 
puters C, through Co, and label the printers P; through P29. 
If we connect Cx to Py fork = 1,2,..., 20 and connect each 
of the computers C2; through C100 to all the printers, then we 
have used a total of 20 + 80-20 = 1620 cables. Clearly this is 
sufficient, because if computers C; through C9 need printers, 
then they can use the printers with the same subscripts, and if 
any computers with higher subscripts need a printer instead of 
one or more of these, then they can use the printers that are not 
being used, because they are connected to all the printers. Now 
we must show that 1619 cables is not enough. Because there 
are 1619 cables and 20 printers, the average number of com- 
puters per printer is 1619/20, which is less than 81. Therefore 
some printer must be connected to fewer than 81 computers. 
That means it is connected to 80 or fewer computers, so there 
are 20 computers that are not connected to it. If those 20 com- 
puters all needed a printer simultaneously, then they would be 
out of luck, because they are connected to at most the 19 other 
printers. 41. Let a; be the number of matches completed by 
hour i. Then 1 < ay < ay < ++: < ays < 125. Also 
25 < ay +24 <a +24 <--- < aj5 + 24 < 149. There 
are 150 numbers a), ..., 475, aq) + 24,...,a75 + 24. By the 
pigeonhole principle, at least two are equal. Because all the 
ajs are distinct and all the (a; + 24)s are distinct, it follows 
that aj = aj + 24 for some i > j. Thus, in the period from 
the (j + 1)st to the ith hour, there are exactly 24 matches. 
43. Use the generalized pigeonhole principle, placing the | S| 
objects f(s) for s € S in |T| boxes, one for each element of 
T. 45. Let d; be jx — N(jx), where N(jx) is the integer 
closest to jx for 1 < j <n. Each d; is an irrational num- 
ber between —1/2 and 1/2. We will assume that n is even; 
the case where n is odd is messier. Consider the n intervals 
{x | j/n <x <(G+D/n}, | -G+ Din <x <—j/n) 
for j = 0, 1,..., (n/2) — 1. If dj belongs to the interval 
{x | 0 < x < 1/n} or to the interval {x | —1/n < x < 0} 
for some j, we are done. If not, because there are n — 2 in- 
tervals and n numbers d;, the pigeonhole principle tells us 
that there is an interval {x | (k — 1)/n < x < k/n}con- 
taining d, and d, with r < s. The proof can be finished by 
showing that (s — r)x is within |/n of its nearest integer. 
47. a) Assume that i, < n for all k. Then by the generalized 
pigeonhole principle, at least [(n2 + 1)/n] = n+ 1 of the 
numbers ij, 72, ...,i,24, are equal. b) If ag, < ax ;,,, then the 
subsequence consisting of ax; followed by the increasing sub- 
sequence of length ix,,, starting at ag,,, contradicts the fact 
that ik; = Uej4- Hence, az; > Akj4)- c) If there is no increas- 
ing subsequence of length greater than n, then parts (a) and 
(b) apply. Therefore, we have ay,,, > ak, > 
a decreasing sequence of length n + 1. 


> dy > ak, 
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Section 6.3 


1. abc, acb, bac, bca, cab, cba 3.720 5.a) 120 
b) 720 c)8 d)6720 e) 40,320 f) 3,628,800 7. 15,120 
9.1320 11.a)210 b)386 c)848 d)252 13. 2(n!)? 
15. 65,780 17.2109 _ 5051 19.a) 1024 ~—b) 45 
c)176 =) 252 -21.a) 120 —Ss bb) 24'—S se) « 120—s—so“d 24 
e)6 f)0 23. 609,638,400 25. a) 94,109,400 b) 941,094 
c) 3,764,376 d) 90,345,024 e) 114,072 f) 2328 g) 24 
h) 79,727,040 i) 3,764,376 ~—j): 109,440 _—- 27. a) 12,650 
b) 303,600 29. a) 37,927 b) 18,915 31. a) 122,523,030 
b) 72,930,375 c) 223,149,655 d) 100,626,625 33. 54,600 


35.45 37.912 39.11,232,000 41.n!/7@M —- ry) 
43.13 45.873 
Section 6.4 


1. x* + 4x3y + 6x7y* + 4xy? + yt 3x8 4 
6x°y + 15xty? + 20x3y? + 15x2y4 + 6xy? + yo 
5.101 7.—-2!0(7) = —94,595,072 9. 7101399 (20) 
11, (= 1) 0-73 (545%) 73) if & = 2 (mod 3) and —100 < 
k < 200; (ottienese 13. 1 9 36 84 126 126 84 
36 9 1 15. The sum of all the positive numbers (7), as 
k runs from 0 to n, is 2”, so each one of them is no big- 


ger than this sum. 17.(7) = ee < 
nt jl 
T 


Tad = a 19. (,"1) + (i) = GK Dn-kF TI 
Ban! = Roce [K+ (n-k+ DI = Werte! a =() ) 
21. a) We show that each side counts the number of ways 
to choose from a set with n elements a subset with k ele- 
ments and a distinguished element of that set. For the left- 
hand side, first choose the k-set (this can be done in (/) ways) 
and then choose one of the k elements in this subset to be 
the distinguished element (this can be done in k ways). For 
the right-hand side, first choose the distinguished element out 
of the entire n-set (this can be done in n ways), and then 
choose the remaining k — 1 elements of the subset from the 
remaining n — | elements ee the set (this can be done in C i) 


n 7a n-(n—1)! _ n—l 

ways). ea peal in. “DL yea ~ n(i1) 
n+ n+l)! 

23. Cy ) = Ka+i-b! — ke Dit =(k= Dy Geer.) 

("yk This identity together with (5 ie = | gives a recursive 


definition. 25. (a) + ea) = eer = +1G3) ia 


Caf] = 3 [Crh +0) = aCe) 27.0 C4) 
counts the number of ways to choose a sequence of r Os and 
n-+ 1 1s by choosing the positions of the Os. Alternately, sup- 
pose that the (j + 1)st term is the last term equal to 1, so that 
n < j <n+r.Once we have determined where the last 1 is, we 
decide where the Os are to be placed in the j spaces before the 
last 1. There aren 1s and j —n Os in this range. By the sum rule 
it follows that there are paces (j j J= saa a) ways to do 
this. b) Let P(r) be the statement to be proved. The basis step 
is the equation (5) = Ci), which is just 1 = 1. Assume that 
P(r) is true. Then 45 ( *) = real: 4 gene | = 


(ntrtty 4" es \=¢ 7 *), using the inductive hypothesis 
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and Pascal’s identity. 29. We can choose the leader first in 
n different ways. We can then choose the rest of the commit- 
tee in 2”! ways. Hence, there are n2”—! ways to choose the 
committee and its leader. Meanwhile, the number of ways to 
select a committee with k people is (i). Once we have cho- 
sen a committee with k people, there are k ways to choose 
its leader. Hence, there are }-7_, k(Z) ways to choose the 
committee and its leader. Hence, )°7_, k(;) = n2"7!. 
31. Let the set have n elements. From Corollary 2 we have 
() — G) + G) -— ++ + CD"G) = 0. It follows that 
(Q+tG)+Q)+--- = ()+G)+(5)+--- - The left-hand side 
gives the number of subsets with an even number of elements, 
and the right-hand side gives the number of subsets with an 
odd number of elements. 33. a) A path of the desired type 
consists of m moves to the right and n moves up. Each such 
path can be represented by a bit string of length m + n with 
m Os and n Is, where a 0 represents a move to the right and a 
1 amove up. b) The number of bit strings of length m + n 
containing exactly n 1s equals (”"'") = (""'") because such a 
string is determined by specifying the positions of the n 1s or 
by specifying the positions of them Os. 35. By Exercise 33 
the number of paths of length n of the type described in that 
exercise equals 2”, the number of bit strings of length 7. On the 
other hand, a path of length n of the type described in Exercise 
33 must end at a point that has n as the sum of its coordinates, 
say (n —k, k) for some k between 0 and n, inclusive. By Exer- 
cise 33, the number of such paths ending at (n — k, k) equals 
Co = (;). Hence, "po (7) = 2". 37. By Exercise 33 
the number of paths from (0, 0) to (n + 1, r) of the type de- 
scribed in that exercise equals ("*"*'). But such a path starts 
by going j steps vertically for some j withO < j < r. The 
number of these paths beginning with 7 vertical steps equals 
the number of paths of the type described in Exercise 33 that 
go from (1, 7) to (7+ 1,7). This is the same as the number of 
such paths that go from (0, 0) to (x, r — j), which by Exercise 
33 equals Ch Because }7’;_o a) — ose ‘eur it 
fliomstia ys CHC"). seat) we) 
c) Ce) d) ( ne 1) e) Largest odd entry in nth row of 


r 
n—-1 La— 


Pascal’s triangle f) cary 


Section 6.5 


1. 243 3.266 5.125 7.35 9.a)1716 b) 50,388 


c) 2,629,575 d) 330 «11.9 13.4,504,501 15. a) 10,626 


b) 1,365 c) 11,649 d)106 17.2,520 19. 302,702,400 
21.3003 23. 7,484,400 25. 30,492 27. C(59, 50) 
29.35 31.83,160 33.63 35.19,635 37.210 

39. 27,720 41. 52!/(7!17!) 43. Approximately 6.5 x 10°? 
45.a) C(k +n—1,n)b)(k +n—1)!/(k—1)! 47. There 
are C(n,n,) ways to choose n; objects for the first box. 
Once these objects are chosen, there are C(n — 71,72) 
ways to choose objects for the second box. Similarly, 
there are C(n — nj — n2, 13) ways to choose objects 
for the third box. Continue in this way until there is 


Cin — ny — ng — +++ — Me-1, MK) = Clg, nk) = 
1 way to choose the objects for the last box (because 
ny +n2z+---+ng = n). By the product rule, the number 
of ways to make the entire assignment is C(n, nj)C(n — 
ni, n2)C(n — ny — no, n3)--- C(n—ny— ng -— +s — 
Ng—1, NK), which equals n!/(n1!n2!---ng!), as straightforward 
simplification shows. 49. a) Because x; < x2 <--- <x;, 
it follows thatxy + 0 < x2 4+ 1 < +: < x +r-—1. 
The inequalities are strict because xj + j —1 < xj41 + j 
as long as xj < xj+41. Because | < xj <n +r — 1, this se- 
quence is made up of r distinct elements from T. b) Suppose 
that] < xy < xo < +++ < x < n+r— 1. Let 
Yk = Xz — (k—1). Then it is not hard to see that yz < yg41 for 
k=1,2,...,r —1andthat 1 < y. <nfork = 1,2,...r. 
It follows that {y1, y2,..., y-} 1s an r-combination with rep- 
etitions allowed of S. c) From parts (a) and (b) it follows 
that there is a one-to-one correspondence of r-combinations 
with repetitions allowed of S and r-combinations of T, a 
set with n + r — 1 elements. We conclude that there are 
C(n + r — 1,1) r-combinations with repetitions allowed 
of S. 51.65 53.65 55.2 57.3 59.a)150 b)25 
c)6 d)2  61.90,720 63. The terms in the expansion are 


of the form x/'1x5? +++ xm", where nj +12 +++: +1m =n. 
Such a term arises from choosing the x; in n, factors, the 
x2 in n2 factors,..., and the x, in nm factors. This can be 
done in C(n; 11, 12, ..., %m) Ways, because a choice is a 
permutation of 1; labels “1,” 12 labels “2,” ..., and, labels 
“m.” 65. 2520 

Section 6.6 


1. 14532, 15432, 21345, 23451, 23514, 31452, 31542, 
43521, 45213, 45321 3. AAAI, AAA2, AABI, AAB2, 
AACI1, AAC2, ABA1, ABA2, ABB1, ABB2, ABC1, ABC2, 
ACAI, ACA2, ACB1, ACB2, ACC1, ACC2, BAAI, BAA2, 
BAB1, BAB2, BACI, BAC2, BBA1, BBA2, BBB1, BBB2, 
BBCI, BBC2, BCAI, BCA2, BCB1, BCB2, BCC1, BCC2, 
CAAI, CAA2, CABI, CAB2, CACI, CAC2, CBAI1, 
CBA2, CBB1, CBB2, CBC1, CBC2, CCA1, CCA2, CCB1, 
CCB2, CCC1, CCC2 5.a)2134  b)54132_ ce) 12534 
d) 45312 =) 7.1234, 1243, 1324, 1342, 1423, 1432, 2134, 
2143, 2314, 2341, 2413, 2431, 3124, 3142, 3214, 3241, 3412, 
3421, 4123, 4132, 4213, 4231, 4312, 4321 9. {1, 2, 3}, 
{1, 2, 4}, {1, 2, 5}, {1, 3, 4}, {1, 3, 5}, 1, 4, 5}, {2, 3, 4}, 
{2, 3, 5}, {2, 4, 5}, {3, 4,5} 11. The bit string representing 
the next larger r-combination must differ from the bit string 
representing the original one in position i because positions 
i+ 1,...,7 are occupied by the largest possible numbers. 
Also a; + 1 is the smallest possible number we can put in 
position i if we want a combination greater than the original 
one. Then a; +2,...,a;+r—i+1 are the smallest allowable 
numbers for positions i + 1 to r. Thus, we have produced 
the next r-combination. 13. 123, 132, 213, 231, 312, 321, 
124, 142, 214, 241, 412, 421, 125, 152, 215, 251, 512, 521, 
134, 143, 314, 341, 413, 431, 135, 153, 315, 351, 513, 531, 
145, 154, 415, 451, 514, 541, 234, 243, 324, 342, 423, 432, 


235, 253, 325, 352, 523, 532, 245, 254, 425, 452, 524, 542, 
345, 354, 435, 453, 534,543 15. We will show that it is a 
bijection by showing that it has an inverse. Given a positive 
integer less than n!, let a), a2, ..., An—1 be its Cantor digits. 
Put n in position n — a,—,; then clearly, a,— 1 is the number 
of integers less than n that follow n in the permutation. Then 
put n — 1 in free position (n — 1) — dy,—2, where we have 
numbered the free positions 1, 2,..., — 1 (excluding the 
position that n is already in). Continue until 1 is placed in 
the only free position left. Because we have constructed an 
inverse, the correspondence is a bijection. 
17. procedure Cantor permutation(n, i: integers with 
n>1land0 <i <n!) 
xi=n 
for j := 1 ton 
pj:=9 
for k := 1ton—1 
c= Lx/a—k)'j;x:=x-—ca—k)h heen 
while p, #0 
h:=h-1 
for j := 1 toc 
h:i=h-1 
while p;, 4 0 
h:=h-1 
Pai=n—-k-+1 
h:=1 
while pr, 4 0 
h:=h+1 
praaA 
{P1P2--- Pn is the permutation corresponding 
to i} 


Supplementary Exercises 


1. a) 151,200 _b) 1,000,000 ¢)210 d)5005 3. 310° 
5. 24,600 7.a) 4060 b) 2688 c) 25,009,600 9. a) 192 
b) 301 c)300 d)300 11.639 13. The maximum pos- 
sible sum is 240, and the minimum possible sum is 15. So 
the number of possible sums is 226. Because there are 252 
subsets with five elements of a set with 10 elements, by 
the pigeonhole principle it follows that at least two have 
the same sum. 15.a)50 b)50 c)14 d) 17 17.Letay, 
a2, ..., Am be the integers, and let d; = a aj. If dj =0 
(mod m) for some i, we are done. Otherwise d; mod m, 
dz mod m, ... , dm mod m are m integers with values in 
{1, 2, ...,m — 1}. By the pigeonhole principle d, = d, 
forsome 1 <k <1 < m. Then Yo g11 4) =d—-d=0 
(mod m). 19. The decimal expansion of the rational num- 
ber a/b can be obtained by division of b into a, where a is 
written with a decimal point and an arbitrarily long string of Os 
following it. The basic step is finding the next digit of the quo- 
tient, namely, [r/b|, where r is the remainder with the next 
digit of the dividend brought down. The current remainder is 
obtained from the previous remainder by subtracting b times 
the previous digit of the quotient. Eventually the dividend has 
nothing but Os to bring down. Furthermore, there are only 
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b possible remainders. Thus, at some point, by the pigeon- 
hole principle, we will have the same situation as had pre- 
viously arisen. From that point onward, the calculation must 
follow the same pattern. In particular, the quotient will re- 
peat. 21. a) 125,970 b) 20 c) 141,120,525 d) 141,120,505 
e) 177,100 f) 141,078,021 23.a)10 b)8 ¢)7 25.3” 
27.Cn+2,r+)=Cn4+ilr+)HD4+Ca4+i1,r)= 
2Cnt+1,r+1)-Cat+1,r4+1+Ca4+1r= 
2Cat+1l,r+)—-(Ca,r+1)+Cta,r)) + (Car) + 
C(tn,r —1))=2C(n4+1,r+1)—Cta,r+1)4+C(n,r—-1) 
29. Substitute x = 1 and y = 3 into the binomial theorem. 
31. Both sides count the number of ways to choose a subset 
of three distinct numbers {i, j,k} withi < j < k from 
{1,2,...,n}. 33.C(n+1,5) 35. 3,491,888,400 37.574 
39.a)45 b)57 c)12 41. a) 386 b)56) 43. 0ifn < m; 
Cin-—1,n—m)ifn>m_ 45.a) 15,625 b) 202 c) 210 
d)10 47. a)3 b)11 c)6 d)10 = 49. There are two pos- 
sibilities: three people seated at one table with everyone else 
sitting alone, which can be done in 2C (n, 3) ways (choose the 
three people and seat them in one of two arrangements), or two 
groups of two people seated together with everyone else sit- 
ting alone, which can be done in 3C (n, 4) ways (choose four 
people and then choose one of the three ways to pair them 
up). Both 2C(n, 3) + 3C(m, 4) and (3n — 1)C(n, 3)/4 equal 
n*/8 — 5n3/12 + 3n?/8 —n/12. 51. The number of per- 
mutations of 2n objects of n different types, two of each type, 
is (2n)!/2”. Because this must be an integer, the denominator 
must divide the numerator. 53. CCGGUCCGAAAG 
55. procedure next permutation(n: positive integer, 

a\,Q2,...,4,: positive integers not exceeding 

n with aja2...a, #nn...n) 

t= 
while a; =n 
q,:i=1 


i:=i-1 
qj :=a,t+1 
{a,a2 ...d, is the next permutation in lexicographic 
order} 

57. We must show that if there are R(m, n — 1) + R(m— 1, n) 
people at a party, then there must be at least m mutual friends 
or mutual enemies. Consider one person; let’s call him Jerry. 
Then there are R(m — 1,n)+ R(m,n—1)—1 other people at 
the party, and by the pigeonhole principle there must be at least 
R(m — 1, n) friends of Jerry or R(m, n — 1) enemies of Jerry 
among these people. First let’s suppose there are R(m — 1, n) 
friends of Jerry. By the definition of R, among these people 
we are guaranteed to find either m — 1 mutual friends or n 
mutual enemies. In the former case, these m — | mutual friends 
together with Jerry are a set of m mutual friends; and in the 
latter case, we have the desired set of m mutual enemies. The 
other situation is similar: Suppose there are R(m, n — 1) ene- 
mies of Jerry; we are guaranteed to find among them either m 
mutual friends or n — 1 mutual enemies. In the former case, 
we have the desired set of m mutual friends, and in the latter 
case, these n — | mutual enemies together with Jerry are a set 
of n mutual enemies. 
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CHAPTER 7 


Section 7.1 


1. 1/13 3.1/2 5.1/2 7.1/64 9.47/52 11. 1/C(52, 5) 
13. 1 — [C(48, 5)/C(52,5)] 15. C13, 2)C(4, 2)C(4, 2) 
C(44, 1)/C(52,5) 17. 10,240/C(52,5) 19. 1,302,540/ 
C(52, 5) 21.1/64 9 23.8/25 25.a)1/ C(50, 6) = 
1/15,890,700  b) 1/C(52, 6) a 1/20,358,520 
c) 1/C(56, 6) = 1/32,468,436 d)1/C(60, 6) = 1/ 
50,063,860 27. a) 139,128/319,865 b) 212, 667/511,313 
c) 151,340/386,529 d) 163,647/446,276 29. 1/C(100, 8) 
31.3/100 33. a) 1/7,880,400 b) 1/8,000,000 
35.a)9/19  b) 81/361 ©) 1/19 d) 1,889,568/2,476,099 
e) 48/361 37.Three dice 39.The door the contestant 
chooses is chosen at random without knowing where the 
prize is, but the door chosen by the host is not chosen at 
random, because he always avoids opening the door with the 
prize. This makes any argument based on symmetry invalid. 
41. a) 671/1296 b) 1 — 3524/3624; noc) The former 


Section 7.2 


1. p(T) = 1/4, p(A) = 3/4 3. pd) = p@) = pG) = 
p(6) = 1/16; p(2) = p(4) = 3/8 5.9/49 7.a)1/2 
b) 1/2 c) 1/3 d) 1/4 e) 1/4 9. a) 1/26! b) 1/26 ce) 1/2 
d) 1/26 e) 1/650 f)1/15,600_ 11. Clearly, p(E U F) => 
P(E) = 0.7. Also, p(E U F) < 1. If we apply Theorem 2 
from Section 7.1, we can rewrite this as p(Z) + p(F) — 
P(E OF) < 1,0r0.74+ 0.5 — p(EN F) < 1. Solv- 
ing for p(E M F) gives p(iEM F) > 0.2. 13. Because 
p(E UF) = p(E) + p(F) — p(EN F) and p(E U F) <1, 
it follows that 1 > p(£) + p(F) — p(E /O F). From this 
inequality we conclude that p(E) + p(F) < 1+ p(EN F). 
15. We will use mathematical induction to prove that the in- 
equality holds forn > 2. Let P(n) be the statement that 
Pj) Ej) < Vint p(Ej). Basis step: P(2) is true be- 
cause p(E, U Er) = p(E\) + p(E2) — p(Ei M £2) < 
P(E) + p(E£2). Inductive step: Assume that P (k) is true. Us- 
ing the basis case and the inductive hypothesis, it follows that 
PUKE Ej) < psa) Ej) + pPExn) < DM) ple). 
This shows that P (k+ 1) is true, completing the proof by math- 
ematical induction. 17. Because E U E is the entire sample 
space S, the event F can be split into two disjoint events: 
F=SOF =(EUE)NF =(ENF)U(EN PF), using the 
distributive law. Therefore, p(F) = p((EN F)U(EN F)) = 
D(E 1 F) + p(E Nn F), because these two events are dis- 
joint. Subtracting p(E M F) from both sides, using the fact 
that p(E 1 F) = p(E) - p(F) (the hypothesis that EF 
and F are independent), and factoring, we have p(F)[1— 
p(E)| = p(E 1 F). Because 1 — p(E) = p(E), this 
says that p(EN F) = p(E)- p(F), as desired. 19. a) 1/12 
bl-P-epe s4e)5 21.614 23.1/4 25. 3/8 
27. a) Not independent b) Not independent c) Not inde- 
pendent 29.3/16 31.a) 1/32 = 0.03125 b)0.49 ~ 


0.02825 c) 0.03795012 33. a) 5/8 b) 0.627649 c) 0.6431 
35. a) p” b) 1 — p” op? tn. pret. ae 
p) @1—[p"+n-p"™!-C—p)] 37. pi, Ei) is 
the sum of p(s) for each outcome s in Ur, E;. Because 
the E;s are pairwise disjoint, this is the sum of the proba- 
bilities of all the outcomes in any of the Es, which is what 
pera p(E;) is. (We can rearrange the summands and still get 
the same answer because this series converges absolutely.) 
39.aE = VU Pi , Fj, so the given inequality now follows 
from Boole’s Inequality (Exercise 15). b) The probability that 
a particular player not in the jth set beats all k of the players 
in the jth set is (1/2) = 2-*. Therefore, the probability 
that this player does not do so is 1 — 2~*, so the probability 
that all m — k of the players not in the jth set are unable to 
boast of a perfect record against everyone in the jth set is 
(1—2-*)"-* That is precisely p(F;). ©) The first inequality 
follows immediately, because all the summands are the same 
and there are (‘7’) of them. If this probability is less than 1, 
then it must be possible that E fails, i.e., that E happens. So 
there is a tournament that meets the conditions of the problem 
as long as the second inequality holds. d) m > 21 fork = 2, 
andm > 91 fork =3 
41. procedure probabilistic prime(n, k) 
composite := false 
i:=0 
while composite = false andi < k 
i:=i+1 
choose b uniformly at random with | <b <n 
apply Miller’s test to base b 
if n fails the test then composite := true 
if composite = true then print (“composite”) 
else print (“probably prime”) 


Section 7.3 


NOTE: In the answers for Section 7.3, all probabili- 
ties given in decimal form are rounded to three decimal 
places. 1.3/5 3.3/4 5.0.481 7.a) 0.999 b) 0.324 
9. a) 0.740 b) 0.260 c¢)0.002 d)0.998 11. 0.724 
13.3/17. 15.a)1/3 b)p(M = j |W =k) = 1 
if i, 7, and k are distinct; ppM = j | W = k) = 0 
ifj = korj = i; p(M = j |W =k) = 1/2 
ifi = kand j #4 i c)2/3  d) You should change 
doors, because you now have a 2/3 chance to win by switch- 
ing. 17. The definition of conditional probability tells us 
that p(F; | FE) = p(EN F;)/p(£). For the numerator, 
again using the definition of conditional probability, we have 
P(E Fj) = p(E | Fj) p(F;), as desired. For the denomina- 
tor, we show that p(E) = )~/_, p(E | F;) p(F;). The events 
EQ F; partition the event E; that is, (EM Fj, )N (EN Fi,) =9 
when i; # iz (because the F;’s are mutually exclusive), and 
LJ!_,(E 2 F;,) = E (because the (J_, F; = S). Therefore, 
P(E) = Vins P(ENF;) = Vin P(E | Fi) p(Fi). 19. No 
21.Yes 23. By Bayes’ theorem, p(S | E, 0 E2) = p(E, 9 
E2 | S)p(S)/[p(E1 9 E2 | S)p(S) + p(E1 9 Ez | S)p(s)]. 


Because we are assuming no prior knowledge about whether 
a message is or is not spam, we set p(S) = p(S) = 0.5, 
and so the equation above simplifies to p(S | E£, NM E2) = 
P(E, 9 E> | S)/[p(E, OE, | S) + p(E, 9 Ep | S)). 
Because of the assumed independence of E;, E2, and S, we 
have p(E, 1M E> | S) = p(E, | S)- p(E2 | S), and similarly 
for S. 


Section 7.4 


1. 2.55 3.5/3 5.336/49 7.170 9.(4n +  6)/3 
11. 50,700,551/10,077,696 ~ 5.03 13.6 15. p(X = 
= Veppe = & = Ved =p pe = 
pA pi! eo — pt = pd — p)P1/-(.— p)) = 
(1 — p)i-! 17.2302 19. (7/2) -7 # 329/12 21.10 
23.1472 pounds 25.p + (n — 1)p(Q1 — p) = 27.5/2 
29.a)0 b)n 31. This is not true. For example, let X be 
the number of heads in one flip of a fair coin, and let Y be 
the number of heads in one flip of a second fair coin. Then 
A(X) + A(Y) = 1 but A(X + Y) = 0.5. 33. a) We 
are told that X; and X2 are independent. To see that X 
and X3 are independent, we enumerate the eight possi- 
bilities for (X 1, X2, X3) and find that (0, 0, 0), (1, 0, 1), 
(0, 1, 1), (1, 1, 0) each have probability 1/4 and the others 
have probability 0 (because of the definition of X3). Thus, 
D(X, = OA X3 = 0) = 1/4, p(X, = 0) = 1/2, and 
p(X3 = 0) = 1/2, so it is true that p(X; = OA X3 = 
0) = p(X, = 0) p(X3 = 0). Essentially the same calculation 
shows that p(X; = 0A X3 = 1) = p(X, = 0)p(X3 = 1), 
P(X, = 1A X3 = 0) = p(X = Ip(X3 = 0), and 
p(X, = 1A X3 = 1) = p(X = 1)p(X3 = 1). There- 
fore by definition, X; and X3 are independent. The same 
reasoning shows that X2 and X3 are independent. To see 
that X3 and X; + X2 are not independent, we observe that 
pP(X%3 = 1AX,4+ X2 = 2) = 0. But p(X3 = I) p(X, + X2 = 
2) = (1/2)(1/4) = 1/8. b) We see from the calculation in 
part (a) that X;, X2, and X3 are all Bernoulli random vari- 
ables, so the variance of each is (1/2)(1/2) = 1/4. Therefore, 
V(X1) + V(X2) + V(X3) = 3/4. We use the calculations 
in part (a) to see that E(X, + X2 + X3) = 3/2, and then 
V(X, + X2 + X3) = 3/4. c) In order to use the first part of 
Theorem 7 to show that V((X,; + X2+---+ Xx) + Xx41) = 
V(X, + X2 -+ Xx) + V(Xx-41) in the inductive step of 
a proof by mathematical induction, we would have to know 
that X; + X2+---+ X, and Xx+1 are independent, but we 
see from part (a) that this is not necessarily true. 35. 1/100 
37. E(X)/a = >), (r/a)- p(X =r) = Vyog 1 P(X =) = 
p(X > a) 39.a) 10/11 b)0.9999 41. a) Each of the 
n! permutations occurs with probability 1/n!, so E(X) is the 
number of comparisons, averaged over all these permutations. 
b) Even if the algorithm continues n — 1 rounds, X will be 
at most n(n — 1)/2. It follows from the formula for expec- 
tation that E(X) < n(n — 1)/2. ce) The algorithm proceeds 
by comparing adjacent elements and then swapping them 
if necessary. Thus, the only way that inverted elements can 
become uninverted is for them to be compared and swapped. 


Answers to Odd-Numbered Exercises S-47 


d) Because X(P) > JI(P) for all P, it follows from the 
definition of expectation that E(X) > E(/). e) This sum- 
mation counts | for every instance of an inversion. f) This 
follows from Theorem 3. g) By Theorem 2 with n = 1, the 
expectation of /;,, is the probability that a, precedes aj; in 
the permutation. This is clearly 1/2 by symmetry. h) The 
summation in part (f) consists of C(n, 2) = n(n — 1)/2 terms, 
each equal to 1/2, so the sum is n(n — 1)/4. i) From part 
(a) and part (b) we know that F(X), the object of interest, is 
at most n(n — 1)/2, and from part (d) and part (h) we know 
that E(X) is at least n(n — 1)/4, both of which are @(n?). 
43.1 45.V(X + Y) = E(X 4+ Y)*?) — E(X + YP = 
E(X? + 2XY + Y?) — [E(X) + E(Y)? = E(X?) + 
2E(XY) + E(Y*) — E(X)? — 2E(X)E(Y) — E(Y) = 
E(X*) — E(X)? + 2[E(XY) — E(X)E(Y)] + EY?) - 
E(Y)? = V(X) + 2Cov(X, Y) + V(Y) 47. [(m — 1)/n]”" 
49. (n —1)"/n™-! 


Supplementary Exercises 


1. 1/109,668 3. a) 1/195,249,054 _ b) 1/5,138,133 
c) 45/357,599 — d) 18,285/18,821 5. a) 1/C(52, 13) 
b) 4/C(52, 13) ¢) 2,944,656/C(52, 13) d) 35,335,872/ 
C(52,13) 7.a)9/2 b)21/4 9.a)9 b)21/2 I1.a)8 
b) 49/6 13.a)n/2"—! b) p(i—p)*~!, where p = 0/2"! 
2 15, SNe Dien! 17a) 2/3 b) 2/3 
19. 1/32 21. a) The probability that one wins 2” dollars 
is 1/2”, because that happens precisely when the player gets 
n — | tails followed by a head. The expected value of the win- 
nings is therefore the sum of 2” times 1/2” as n goes from | to 
infinity. Because each of these terms is 1, the sum is infinite. 
In other words, one should be willing to wager any amount of 
money and expect to come out ahead in the long run. b) $9, $9 
23. a) 1/3 when S = {1, 2, 3, 4,5, 6, 7, 8, 9, 10, 11, 12}, 
A = {l, 2,3, 4,5, 6, 7, 8,9}, and B = {1, 2, 3, 4}; 


1/12 when S = (I, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}, 
A = 14,5, 67,8; 9, 10, 11, 12), and B= (1) 2,3,4) 
b)1 when S = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}, 
A = {4,5, 6, 7, 8,9, 10, 11, 12}, and B = {1, 2, 3, 4}; 
3/4 when S = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}, 
A = (Il, 2, 3, 4,5, 6, 7, 8,9}, and B = {1, 2, 3, 4} 
25.a) p(E1 NO E2) = p(E1)p(E2), p(Ai £3) = 


P(E) p(E3), p(E20 E3) = p(E2)p(E3), p(EiN £20 £3) = 
P(E\) p(E2) p(E3) b) Yes c) Yes; yes d) Yes;no e) 2”—n—1 
27. a) 1/2 under first interpretation; 1/3 under second inter- 
pretation b) Let M be the event that both of Mr. Smith’s 
children are boys and let B be the event that Mr. Smith chose 
a boy for today’s walk. Then p(M) = 1/4, p(B | M) = 1, 
and p(B | M) = 1/3. Apply Bayes’ theorem to compute 
pD(M | B) = 1/2. e) This variation is equivalent to the 
second interpretation discussed in part (a), so the answer is 
unambiguously 1/3. 29. ViaX +b) = E((aX + b)*) — 
E(aX +b)? = E(a*X? + 2abX + b?) — [aE(X) + bf = 
E(a?X?)+E (2abX)+E (b?)—[a? E(X)?+2ab E(X)+b?] = 
a’ E(X*) + 2abE(X) + b* — a? E(X)* — 2abE(X) —b* = 
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a?[E(X”) — E(X)*] = a*V(X)  31.To count every el- 
ement in the sample space exactly once, we must include 
every element in each of the sets and then take away the 
double counting of the elements in the intersections. Thus 
P(E, UV EQU---U En) = pC) + p(E2) +--+ + p(Em) — 
P(E\O Ex) — p(E, 1 E3)—---— p(E,N Em) — p(E2N E3) — 
P(E2  E4) +++ — p(E2 ON Em) P(Em-1 Em) = 
qm — (m(m — 1)/2)r, because C(m, 2) terms are being 
subtracted. But p(E; U E2 U---U E,) = 1, so we have 
qm—[m(m—1)/2]r = 1. Becauser > 0, this equation tells us 
that gm > 1, so q => 1/m. Because q < 1, this equation also 
implies that [m(m — 1)/2]r = qm —1 < m—1, from which it 
follows thatr <2/m. 33. a) We purchase the cards until we 
have gotten one of each type. That means we have purchased 
X cards in all. On the other hand, that also means that we 
purchased Xo cards until we got the first type we got, and then 
purchased X; more cards until we got the second type we got, 
and so on. Thus, X is the sum of the X;’s. b) Once j distinct 
types have been obtained, there are n — j new types available 
out of a total of n types available. Because it is equally likely 
that we get each type, the probability of success on the next 
purchase (getting a new type) is (n — j)/n. c) This follows 
immediately from the definition of geometric distribution, the 
definition of X ;, and part (b). d) From part (c) it follows that 
E(X;) =n/(n — j). Thus by the linearity of expectation and 
part (a), we have E(X) = E(Xo) + E(X1) +--- + E(Xn-1) 
= Be te tt aen(t+ te +4), e) About 
224.46 35.24. 134/(52-51- 50-49) 


CHAPTER 8 


Section 8.1 


1. Let P(n) be “H,, = 2” —1.” Basis step: P (1) is true because 
A = 1. Inductive step: Assume that H, = 2” — 1. Then be- 
cause H,+1 = 2H, +1, it follows that Hy41 = 2(2"—1)+1= 
2+1 _ 1, 3.a) dn = 2an-1 + ans forn > 5 b)ag = 1, 
a 2, a2 4, a3 8, a4 16 ©) 1217 5.9494 
7.) dy =An—1 tn_-2 + 2"7? forn > 2 b) ay = 0, a; =0 
c)94 9. a) dy = An—-1 +Qn-2 + Qn_-3 forn > 3 b) ag = 1, 
a, =2,a2=4 ©)81 Il.a)a, =a,_-| + q_2 forn > 2 
b) ap = 1,aqy = 1 ©)34) 13. a) ay) = 2a,_1 + 2a,_2 for 
n>2 b)ag=1,a; =3 c€)448 15. a) ay = 2ayn_1 + ay_2 
forn >2 b)ag=1,a,; =3 ¢) 239 17. a) dy = 2ay,_ for 
n>2 b)ay =3 ¢)96 19. a) ay = dy_1 +ay_2 forn > 2 
b) ap = lay = 1 ©) 89 21.a)R, =n+R,-1, Ro = 1 
b) Ry =n(n+1)/2+1 23. a) Sy = Sp_ + (n? —n +2)/2, 
So = 1 b)S, = W +5n+6)/6 25.64 27. a) dn = 
2an—1 + 2an—2 b) ao = 1,a; = 3 ©) 1224 29. Clearly, 
S(m,1) = 1 form > 1. If m > _ n, then a function that 
is not onto from the set with m elements to the set with n 
elements can be specified by picking the size of the range, 
which is an integer between 1 and n — | inclusive, picking 
the elements of the range, which can be done in C (n, k) ways, 
and picking an onto function onto the range, which can be 


done in S(m, k) ways. Hence, there are 2s C(n, k)S(m, k) 
func- tions that are not onto. But there are n” functions 
altogether, so S(m,n) = n™ — S771 Cn, Sm, k). 
3l.a) C5 = CoCa + C1 C3 4+ C2C2 + C3C, + CsaCo = 
1-144+1-54+2-2+4+5-14+14-1=42 b) C(10,5)/6 = 42 
33. J) = 1, /@2) = 1, JG) = 3, J4 = 1, JO) = 3, 
J(6) = 5, J(7) = 7, J(8) = 1, JY) = 3, JO) = 5, 
J(11) =7, J(12) = 9, J(13) = 11, J4) = 13, 5) = 15, 
J(16) = 1 ‘35. First, suppose that the number of people is 
even, say 2n. After going around the circle once and returning 
to the first person, because the people at locations with even 
numbers have been eliminated, there are exactly n people left 
and the person currently at location 7 is the person who was 
originally at location 27 — 1. Therefore, the survivor [originally 
in location J (2n)] is now in location J (7); this was the person 
who was at location 2/(n) — 1. Hence, J(2n) = 2J(n) — 1. 
Similarly, when there are an odd number of people, say 2n+ 1, 
then after going around the circle once and then eliminating 
person |, there are n people left and the person currently at lo- 
cation i is the person who was at location 27+ 1. Therefore, the 
survivor will be the player currently occupying location J(n), 
namely, the person who was originally at location 2J(n) + 1. 
Hence, J(2n + 1) = 2/(n) + 1. The basis step is J(1) = 1. 
37. 73, 977, 3617 39. These nine moves solve the puzzle: 
Move disk | from peg | to peg 2; move disk 2 from peg | to 
peg 3; move disk 1 from peg 2 to peg 3; move disk 3 from 
peg | to peg 2; move disk 4 from peg | to peg 4; move disk 3 
from peg 2 to peg 4; move disk | from peg 3 to peg 2; move 
disk 2 from peg 3 to peg 4; move disk | from peg 2 to peg 4. 
To see that at least nine moves are required, first note that at 
least seven moves are required no matter how many pegs are 
present: three to unstack the disks, one to move the largest 
disk 4, and three more moves to restack them. At least two 
other moves are needed, because to move disk 4 from peg 1 
to peg 4 the other three disks must be on pegs 2 and 3, so at 
least one move is needed to restack them and one move to 
unstack them. 41. The base cases are obvious. Ifn > 1, 
the algorithm consists of three stages. In the first stage, by the 
inductive hypothesis, R(m — k) moves are used to transfer the 
smallest n — k disks to peg 2. Then using the usual three-peg 
Tower of Hanoi algorithm, it takes 2* _ 1 moves to transfer the 
rest of the disks (the largest k disks) to peg 4, avoiding peg 2. 
Then again by the inductive hypothesis, it takes R(n — k) 
moves to transfer the smallest n — k disks to peg 4; all the 
pegs are available for this, because the largest disks, now on 
peg 4, do not interfere. This establishes the recurrence rela- 
tion. 43. First note that R(n) = VaR) -RG-D) 
[which follows because the sum is telescoping and R(0) = 0]. 
By Exercise 42, this is the sum of 2*'-! for this range of val- 
ues of 7. Therefore, the sum is ae i2'-|, except that if 
n is not a triangular number, then the last few values when 
i = k are missing, and that is what the final term in the 
given expression accounts for. 45. By Exercise 43, R(n) is 
no larger than sae i2'—! Itcan be shown that this sum equals 
(k+1)2* —24+!41, so itis no greater than (k+ 1)2*. Because 
n > k(k—1)/2, the quadratic formula can be used to show that 


k <1+-J/2n foralln >_1. Therefore, R(n) is bounded above 
by (1 + V2n + 1)2!+¥2" < 8./n2V2" for all n > 2. Hence, 
R(n) is O(/n2¥"), 47.a)0 b)O c)2 d) 27-1! — 2"-2 
49. An—2V an t+V7 an = An —2(Ay —Gn—1) + (Van — Van—1) = 
=An + 2an-1 + [Gn an-1) (Qn—-1 an-2)] = 
=n + 2an—1 + (Gn — 2an—1 + Qn-2) = Qn—-2 Sle ay = 
Gn—1 + An—2 = (An — Van) + (Qn — 2Van + Van) = 
2an — 3Vady + V2dn, OF dn = 3Vay — V2a, 53. Insert 
S(O) := @ after T(0) := O (where S(j) will record the 
optimal set of talks among the first 7 talks), and replace the 
statement T(j) := max(w; + T(p(j)), TV — 1)) with the 
following code: 
ifw; +T(p(j)) > TG — 1) then 
T(j) = Wj + Tp) 
S(j) = SPU) U Li} 


else 
TA) :=TU-D 
S(j) := SG —-1) 


55. a) Talks 1, 3, and 7 b) Talks | and 6, or talks 1, 3, 
and 7 c) Talks 1,3, and 7 d) Talks 1 and6 57. a) This 
follows immediately from Example 5 and Exercise 41c in 
Section 8.4. b) The last step in computing Aj; is to mul- 
tiply Ajx by Axy1,; for some k between i and j — | in- 
clusive, which will require mjm x4 1mj+1 integer multiplica- 
tions, independent of the manner in which Ajx and Ag+1,j 
are computed. Therefore to minimize the total number of 
integer multiplications, each of those two factors must be 
computed in the most efficient manner. c) This follows im- 
mediately from part (b) and the definition of M(i, /). 
d) procedure matrix order(m\,...,M™n+1: 
positive integers) 
fori :=1ton 
M(i,i):=0 
ford :=1ton—1 
fori :=1ton—d 
min := 0) 
fork :=itoi+d 
new := M(i,k) + M(k+1,i+d)+mjmgyimisa+i 
if new < min then 
min := new 
where(i,i +d) :=k 
M(i,i+d):= min 


e) The algorithm has three nested loops, each of which is in- 
dexed over at most 7 values. 


Section 8.2 

1. a) Degree 3 b)No c) Degree 4 d) No e) No f) Degree 

2 g)No 3.a)aq, = 3-2" bD)aq = 2 Oq = 

3-2" —2.3" d)a, =6-2"—2-n2" e)a, =n(—2)""! 

Hag = 2 = (-2" ga, = (1/2)? = G1! 
4 1 V5 n+l 1 1-5 n+l 

5. an = Fe(454)"- (54) 7.2 +C-0"1/3 


9.a) Py = 1.2Py-1 + 0.45P,-2, Po = 100,000, P} = 
120,000 b) P, = (250,000/3)(3/2)"+(50,000/3)(—3/10)” 
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11. a) Basis step: Forn = 1 wehave 1 = 0+ 1, and 
forn = 2 we have 3 = 1 +4 2. Inductive step: As- 
sume true fork < n. Then Lypyy = Lp + Ln-1 = 
Sn it fn4it fn at fn=Chh it fn 2+ (fn4it fr) = 
n n 

i+. WI, = (434) i (54) ia, = 
8(—1)" — 3(-2)? + 4-3") 15.a, = 5 + 3(—2)" — 3” 
17. Leta, = C(n, 0) + C(n—1, 1) +---+C(n—k, k) where 
k = |n/2]. First, assume that n is even, so that k = n/2, 
and the last term is C(k,k). By Pascal’s identity we have 
a =1+C(n—2,0)+C(n—-2,1)+C(n—3,1)+C(n 
3,2)+---+¢Cn—k,k-2)+CH—k,k -DYD+1= 
1+C(n—2, 1)+C(n—3, 2)+---+Ca—k,k-)4+Ca—- 
2,0) +C(n—3, I)+---+Cn—-k,k-—2)4+1 = ay_-1 +ayn_2 
because |(n 1)/2] =k 1 = |(m — 2)/2]. A sim- 
ilar calculation works when n is odd. Hence, {a,} satisfies 
the recurrence relation a, = dy,—1 + dy—2 for all posi- 
tive integers n,n > 2. Also, a; = C(1,0) = 1 and 
az = C(2,0) + CU, 1) = 2, which are fo and fs. It 
follows that an = fn+1 for all positive integersn. 19. dy = 
(n?+3n+5)(-1)" 21. (aio +a1,1n +a1,2n? +.41,3n7) + 
(a2,9 +.a2,1n +a2,2n”)(—2)" + (a3,9 +.43,11)3” + a4,o(—4)" 
23. a) 3a,_) + 2” = 3(-2)? + 2? = 2°(-34+1) = 
ont = dp b) dn = a3? — gn+l c) dn = antl _ gut 
23.ayA = -1,B = -7 bay = a2” —n-7 
¢C) dy = 11-2" —n—7 27. a) p3n? + pon? + pin + po 
b) n? po(—2)"_ ¢) n*(pin + po)2" d) (p2n* + pin + po)4” 
e)n?(pyn? + pin + po)(—2)" — f)n?(pan* + p3n3 + 
pon + pin + po)2"” g) po 29.a)a, = a2" + 37+! 
b)an = —2-2° +3" 31a, = a2" + B3" —n- 
atl 4 3n/2421/4 33.a, = (a+ Bn+n? + n3/6)2" 
35.a, = —4- 2" — n?/4 — 5n/2 + 1/8 + (39/8)3" 
37.d, =n(n+1)(1+2)/6 39.a)1,—-1,i,-i b) a, = 
i z( 1)" + 24i jn + 21 ( i)" 41. a) Using the formula 
f 1 (444) -| 1 (54) 
ns 2 V5\ 2 
1 [V5 < 1/2. This means that f, is the integer clos- 
est to zz ae . b)Less when n is even; greater 
when n is odd 43. ay = fn-1 + 2fn - 1 
45.a) dyn = 3an-1 + 4an-2,a09 = 2,aq, = 6 Dan = 
(47tt 4 (-1)"]/5 47. a) ay = 2an41 + (n — 1)10,000 
b) a, = 70,000 - 2”-! — 10,000n — 10,000 49.a, = 
5n?/12 + 13n/12 + 1 51. See Chapter 11, Section 5 in 
[Ma93]. 53.67-4"—!/n 


for fn, we see that < 


Section 8.3 


1. 14 3.The first step is (1110)2(1010)2 = (24 4+ 
2?)(11)2 (10)2 + 272 — (10)2][(10)2 — (10)2] + 
(27 + 1)(0)2 (10)2. The product is (10001100)o. 
5.C = 50, 665C + 729 = 33,979 7.a)2 b)4 
c)7 9%. a)79_ b) 48,829 c) 30,517,579 11. O(log n) 
13. O(n!$832) 15.5 17. a) Basis step: If the sequence has 
just one element, then the one person on the list is the winner. 
Recursive step: Divide the list into two parts—the first half 
and the second half—as equally as possible. Apply the algo- 
rithm recursively to each half to come up with at most two 


S-50 Answers to Odd-Numbered Exercises 


names. Then run through the entire list to count the number of 
occurrences of each of those names to decide which, if either, 
is the winner. b) O(nlogn) 19.a) f(n) = f(n/2) +2 
b) Odogn) 21.a)7 b) O(logn) 


23. a) procedure largest sum(a,,..., dn) 
best := 0 {empty subsequence has sum 0} 
fori := 1ton 
sum := 0 


for j:=i+1ton 
sum := sum + aj 
if sum > best then best := sum 
{best is the maximum possible sum of numbers 
in the list} 

b) O(n?) c) We divide the list into a first half and a second 
half and apply the algorithm recursively to find the largest 
sum of consecutive terms for each half. The largest sum of 
consecutive terms in the entire sequence is either one of these 
two numbers or the sum of a sequence of consecutive terms 
that crosses the middle of the list. To find the largest possi- 
ble sum of a sequence of consecutive terms that crosses the 
middle of the list, we start at the middle and move forward 
to find the largest possible sum in the second half of the list, 
and move backward to find the largest possible sum in the 
first half of the list; the desired sum is the sum of these two 
quantities. The final answer is then the largest of this sum 
and the two answers obtained recursively. The base case is 
that the largest sum of a sequence of one term is the larger of 
that number and 0. d) 11,9, 14 e) S(m) = 2S(n/2) +n, 
Cin) = 2C(n/2) +n + 2, SA) = 0,CU) = 1 
f) O(n log n), better than O(n?) 25. (1, 6) and (3, 6) at 
distance 2. 27. The algorithm is essentially the same as the 
algorithm given in Example 12. The central strip still has width 
2d but we need to consider just two boxes of size d x d rather 
than eight boxes of size (d/2) x (d/2). The recurrence relation 
is the same as the recurrence relation in Example 12, except 
that the coefficient 7 isreplacedby 1. 29. Withk = log, n, it 
follows that f(n) = a* f (1) + hha! e(n/b!)4 = ak fA) + 
Yt ent = ak (1) + ken = a8" f (1) + c(log, nynt = 
n'8o4 £1) + cn4 log,n = né f (1) + cnt log, n. 31. Let 
k = log, n where n is a power of b. Basis step: lfn = 1 
and k = 0, then cin? + conS4 = cy tO = b¢c/ 
(b¢ — a) + f(1) + b¢c/(a — b“) = f (1). Inductive step: 
Assume true for k, where n = b*. Then forn = bk+!, f@m= 
af (n/b) + cn4 = a{[b4c/(b4 — a)\(n/b)* + fA) + b4%e/ 
(a — b*)] - (n/b)©8%)} + cn? = b4c/(b4 — a)n4a/b4 + 
Lf) + b4c/(a — b4)n!84 + cnt = n@[ac/(b4 —a)+ 
c(b4 — a)/(b4 — a)] + Lf) + bfe/@ — b4c)n&4 = 
[b¢c/(b4 — an? + [f (1) + b¢e/(a — b®)Jn'%*, 33. IF 
a> bé , then log, a > d, so the second term dominates, 
giving O(n'8%), 35. O(n!?24>) 37. O(n) 


Section 8.4 


1. f(x) =2(¢8-1)/(x-1) 3. a) f(x) = 2x — x) /(I- 
x) b)x3/(1—x) e)x/(1— x3) d)2/(1—2x) e) (1+ x)’ 
f) 2/(1+x) g) [1/(L—x)]—x? h) x3/(1—x)? 5. a) 5/(1—x) 
b) 1/(1—3x) ©) 2x3/(1—x) d) G—x)/ (1—x)? e) 1+x)8 
7.a) dag = —64, a, = 144, az = —108, a3 = 27, anda, = 0 
for alln > 4 b) The only nonzero coefficients are ag = 1, 
a3 = 3,a6 = 3,a9 = 1. C)a, = 5" d) ay = (—3)""? for 
n > 3,andag =a; = d2 = 0 e@) ap = 8, a, = 3, a = 2, 
dy = O for odd n greater than 2 and a, = 1 for even n greater 
than2 f) a, = 1ifnisapositive multiple 4, a, = —lifn < 4, 
anda, = 0 otherwise g) a, =n—1forn > 2andap = a; = 0 
h) dn = 2"*!/n! 9.a)6b)30)9d)0 e)5_ 11. a) 1024 
b) 11 c) 66 d) 292,864 e) 20,412 13.10 15.50 17.20 
19. f(x) = If — x) — x?) Gd — x)d — x!) 
21.15 23.a)xt(1 + x + x? + x7)?/ (1 — x) b) 6 
25. a) The coefficient of x” in the power series expansion of 
1/[( —x3) — x4) — x?°)] by 1/( — x3 — x4 — x9) &)7 
d) 3224 927. a)3 b) 29 c) 29 d) 242 = 29. a) 10 b) 49 c)2 
d)4 31. a) G(x) — ap — ayx — ax? _b) G(x?) ©) x*G(x) 


d) G(2x) e) i G(t)dt f) G(x)/(1—x) 33. ay = 2-3-1 


35. a, = 18-3 -—12-2* 37. ay =k? +8k+204 (6k—18)2* 
39. Let G(x) = yah fuxk. After shifting indices of summa- 
tion and adding series, we see that G(x) —xG(x) x2G(x) a 
fo + (fi — fox + VRofe — fret — fe-2)x* = 
O+x+ Dp, Ox*. Hence, G(x) — xG(x) — x?G(x) = 
x. Solving for G(x) gives G(x) = x/(1 — x — x?). 
By the method of partial fractions, it can be shown that 
x/Q —x — x7) = (1/V5)[1/ — ax) — 1/0 — px], 
where a = (1 + /5)/2 and 6 = (1 — V5)/2. Using the 
fact that 1/(1 — ax) = a x*, it follows that G(x) = 
(1/V5)- ola — Bk) x*. Hence, fe = (1/75): (ak — BX). 
41. a)Let G(x) = yee 5 C,x" be the generating func- 
tion for {Cy}. Then G(x)? = 7 9(D fg CeCn—x) x” = 


yO A Gat = Ye Ca. Hone, 
xG(x)? = YO, Cyx”", which implies that xG(x)? — 
G(x) + 1 = 0. Applying the quadratic formula shows that 
G(x) = a pak . We choose the minus sign in this for- 


mula because the choice of the plus sign leads to a division 
by zero. b) By Exercise 40, (1 — 4x)~/? =v) Cee. 
Integrating term by term (which is valid by a theorem from cal- 


culus) shows that /j (1 —4t)-dt =F , ar! = 


areas at (7")x". Because fj (1—4t)~!/2de = Eve = 
xG(x), equating coefficients shows that C, = a"). 
c) Verify the basis step forn = 1, 2, 3, 4, 5. Assume 
the inductive hypothesis that C; = 2i-! forl < j < 
n, wheren > 6. Then Cy = Sf29 CkCn-k-1 = 

Fai CeCn—e-1 > ( — 2)2k-1 9"? = (nn — 2)2"-1/4 > 
2"-!_ 43. Applying the binomial theorem to the equality 
(1+ xy" = (1+ x)"(1 +x)", shows that "0" Cm + 
n, r)x'= yoy Cm, rx’ V9 Cn, r) x” - 
ey [ieo C (m, r — k) C (n, k)] x”. Comparing coeffi- 
cients gives the desired identity. 45.a)2e* b)e* oc) e* 
d)xeX + & 47.a)aq, = (-1)" Da = 3. 2" 


c)a, = 3" —3-2” d)a, = (—2)" forn > 2,a, = —3, 
ao = 2 ea, = (-2)" +n! fa, = (—3)" +n! - 2” 
forn > 2,a9 = lay = —2 g)a, = Oifn is odd and 
dy = ni/(n/2)! if nis even 49.a)a, = 6ay_; + 8"7! 
forn > 1,a) = 1 b) The general solution of the associ- 
ated linear homogeneous recurrence relation is a) = a6". 
A particular solution is aw? = 5 - 8”. Hence, the general 
solution is da, = a6” + 5 - 8”. Using the initial condition, 
it follows thata = 5 Hence, ad, = (6" + 8")/2. c) Let 
G(x) = aaa arx*, Using the recurrence relation for {a,x}, it 
can be shown that G(x)—6x G(x) = (1—7x)/(1—8x). Hence, 
G(x) = (1 —7x)/[C — 6x)(1 — 8x)]. Using partial fractions, 
it follows that G(x) = (1/2)/(. — 6x) + (1/2)/(1 — 8x). 
With the help of Table 1, it follows that a, = (6” + 8”)/2. 
Shes poe tio 53 + x) +x)? +-2)7-- 
55. The generating functions obtained in Exercises 52 and 53 
are equal because (1 +x)(1+x7)(1+x3)-.-= i . = . 
tease = hy gts pt 57. a) Gx (1) 
Spt Hh) TF = Pa HHH 1 HG = 
fey PX Hh) | xa = Deg P(X = bk" 
Do P(X = Hk = E(X) GE = BYR y p(X = 
k) + x*|yar = Deo P(X = kW) - k(k — 1) - x* [21 = 
yy D(X =k): (k?—k) = V(X) + E(X)* — E(X). Combin- 
ing this with part (b) gives the desired results. 59. a) G(x) = 
p™/(1—qx)" b) V(x) = mq/p? 


Section 8.5 


l.a) 30 b) 29c)24d)18 3.1%  5.a)300 b) 150 ce) 175 
d) 100 7.492 9.974 11.55 13.248 15.50,138 17.234 
19. |Ay UA2U A3 UA4U A5| = |A1|+|A2|+]A3|+]Aa] + 
|As|—|A1N Aa|—|A1N A3| —|A1 Ag] — |A1NAs|—|A2n 
A3|—|A2N Ag| —|A2N As5| — |A3M Ag] — |A3NA5|—|AgN 
As|+|A1NA2NA3|+|A1NA2N Ag|+|A1NA2NAs5|+]A1N 
A3NAqg|+|A1NA3NA5|+|A,N AGN As5|+]A2NAZN Ag|+ 
|JA2NA3NA5|+|A2N AsN As|+|A3N A4N As5|—|A,NA2N 
A3MAq4|—|A1)N A2N A3NA5|—|A,NA2N AgN As5|—|AIN 
A3N A4NAs|—|A2NA3N A4gNAs|+|A1NA2NAZ3N A4N AS 
21. |AyUA2UA3UAqUA5UA6| = |A1|+]A2|+]A3]+]Aq|+ 
|A5|+|A6|—|A1N A2|—|A19A3|—|A1N Ag] —|A1NAs5|— 
|A{NA6|—|A2MA3|—|A2NAg|—|A2NA5|—|A2MA6|—|A3N 
A4|—|A3NAs5|—|A3N Ao| —|A4N As5| —|A4N Ao| —|AsN Ao 
23. p(E) U E2 U E3) = p(E1) + p(E2) + p(E3) — p(Ei on 
Ex) — p(E, 1 E3) — p(E2 9 E3) + p(E, 9 E29 E3) 
25. 4972/71,295 27. p(E,; U Eo U £3 U Ey U Es) = 
P(E\) + p(E2) + p(E3) + p(E4) + p(Es) — p( E19 E2) — 
P(E\NE3)— p(E\ 1 £4) — p(E\N Es) — p( E20 E3)— p(£2n 
E4)— p(E20 Es) — p( £30 E4)— p(E3N Es) — p(E4N Es) + 
P(E\N E20 E3)+p( Ein £20 E4)+ p(EiN E2N Es)+ p( Ein 
E30 E4)+ p(E\O E30 Es)+ p(EiN E4gN Es) + p(E£20 E3M 
E4)+ P(E2N E30 Es)+ p( E20 Eg Es) + p(E3N E4n Es) 
29. p (Uj=1 Ei) = VisienP(ED — VicicjenP(EiNEj)+ 
eee PLEIN Ej ON Ex) +--+ (Pep (om E;) 
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Section 8.6 


1.75 3.6 5.46 7.9875 9.540 11.2100 13.1854 
15. a) Dj09/100! b) 100.Do9/100! c) C(100,2)/100! 
d)O e)1/100! 17. 2,170,680 19. By Exercise 18 we 
have D, — nDy-| = [Dn-1 (n 1) Dyj—2]. Iterat- 
ing, we have D, — nDn—-» = —[Dy-1—(n—-1)Dn-2] = 
—[-—(Dn-2 — (n — 2)Dn—3)] = Dn-2 — (n — 2)Dn-3 = 
s+. = (-1)"(D2 — 2D,) = (—1)” because D2 = | and 
D,=0. 21.Whennisodd 23. @(n) =n— vr, a + 


n__ _ yyy” (a-+) 25.4 


icin or aed P1 P2°**Pm i=l Pi 

27. There are n™ functions from a set with m elements to a 
set with n elements, C(n, 1)(n — 1)” functions from a set 
with m elements to a set with n elements that miss exactly 
one element, C(n, 2)(n — 2)" functions from a set with m 
elements to a set with n elements that miss exactly two el- 
ements, and so on, with C(n,n — 1) - 1’ functions from a 
set with m elements to a set with n elements that miss exactly 
n—1lelements. Hence, by the principle of inclusion—exclusion, 
there are n™” — C(n, 1)(n — 1)" + C(, 2)(n — 2)” — + + 
(—1)""!C(n, n — 1)- 1” onto functions. 


Supplementary Exercises 


ld. a)An = 4An-1 b)A1 = 40 c)An = 10. 4” 
3.a) M, = M,-1 + 160,000 b) M; = 186,000 c) M, = 
160,000n + 26,000 d) 7, = T,-1 + 160,000n + 26,000 
e) T, = 80,000n? + 106,000n 5.a) ay = an—2 + ay—3 
b) a, = 0,a2 = 1,a3 =1 c)ajg=12 7.a)2 b)5 ©)8 
d)16 9.a, = 2") lea, = 24+ 4n/3 + n?/2 + n3/6 
13. a, = dn—-2 + a,-3 15. a) Under the given conditions, 
one longest common subsequence clearly ends at the last term 
in each sequence, 80 dm = by = Cp. Furthermore, a longest 
common subsequence of what is left of the a-sequence and the 
b-sequence after those last terms are deleted has to form the 
beginning of a longest common subsequence of the original 
sequences. b) If cp 4 dm, then the longest common subse- 
quence’s appearance in the a-sequence must terminate before 
the end; therefore the c-sequence must be a longest common 
subsequence of dj, a2, ..., Am—1 and by, bo, ..., by. The 
other half is similar. 
17. procedure howlong(a,...,Qn,01,..- 
for i := 1 tom 
L(i, 0) :=0 
for j :=1ton 
L(O, 7) :=0 
for i := 1 tom 
for j :=1ton 
ifa; = b; then Li, 7) :=LG—-1,j7-)+1 
else L(i, j) := max(L(i, 7 — 1), LG — 1, j)) 
return L(m,n) 


, by: Sequences) 
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19. f(n) = (4n? —1)/3 21. O(n*) 23. O(n) 25. Using 
just two comparisons, the algorithm is able to narrow the 
search for m down to the first half or the second half of the 
original sequence. Since the length of the sequence is cut in 
half each time, only about 2 log, n comparisons are needed 
in all. 27.a) 182 + 18 b)18 oc)O0 29. A(anby) = 
Gnt1Ont1— Anbn = Anti (nti — bn) + bn(Qn41 — Gn) = 
anti Aby + by Aa, 31. a) Let Gx) = 79 ayx". Then 
G' (x) = yo nanx"—! = TW on + Dan+1x". Therefore, 
G'(x)—G(x) = rol (nt Dan41—anJx" = Yo, 29 x" /n! = 
e*, as desired. That G(0) = ay = | is given. b) We have 
[e~* G(x)]' = e* G' (x) —e7* G(x) = e 7 [G' (x) — G(x)] = 
e-*-e* = 1. Hence, e-*G(x) = x +c, where c is a constant. 
Consequently, G(x) = xe* + ce*. Because G(O) = 1, it 
follows thatc = 1. ¢) We have G(x) = yx" 1/n!+ 
Yor xt /nl = Oo x" /(n — I)!+ Oy x” /n!. Therefore, 
a = 1/(n—1)!+ 1/n! foralln > 1,andag = 1. 33.7 
35.110 37.0 39.a)19 b)65 c¢)122 d)167 e) 168 
41. Dy-1/(a— 1)! 43. 11/32 


CHAPTER 9 


Section 9.1 


1. a) {(0, 0), CG, 1), @, 2), GB, 3)} bI{G, 3), @, 2), 
(3, 1), 4,0)} ©) {C, 0), 2, 0), 2, D, G, 0), GB, 1), G, 2), 
(4, 0), (4, D, (4, 2), 4, 3)} d) {, 0), G, D, C1, 2), CL, 3), 
(2, 0), (2, 2), (3, 0), (3, 3), (4,0)} &) {, D, C1, 9), CL, 1), 
(, 2), Cd, 3), 2, D, @, 3), 3, D, G, 2), 4 D, 4, 3)} 
f) {(, 2), (2, 1), (2, 2)} 3. a) Transitive — b) Reflexive, 
symmetric, transitive ¢)Symmetric d) Antisymmetric 
e) Reflexive, symmetric, antisymmetric, transitive 
f)None of these properties 5. a) Reflexive,  tran- 
sitive b) Symmetric c) Symmetric d) Symmetric 
7.a)Symmetric b) Symmetric, transitive c) Symmetric 
d) Reflexive, symmetric, transitive e) Reflexive, transi- 
tive f) Reflexive, symmetric, transitive g) Antisymmetric 
h) Antisymmetric, transitive 9. Each of the three properties 
is vacuously satisfied. 11. (c),(d),(f) 13. a) Not irreflex- 
ive b) Not irreflexive c) Not irreflexive d) Not irreflexive 
15. Yes, for instance {(1, 1)} on {1,2} 17.(a,b) € R 
if and only if a is taller than b 19.(a) 21.None 
23.VaVb (a,b) € R —> (b,a) € RJ 25.2" 
27. a) {(a, b) | b divides a} b) {(a, b) | a does not divide 
b} 29. The graph of f~! 31. a) {(a, b) | a is required to 
read or has read b}_ b) {(a, b) | a is required to read and has 
read b} c) {(a, b) | either a is required to read b but has not 
read it or a has read b but is not required to} d) {(a, b) | a 
is required to read b but has not read it} e) {(a, b) | a has 
read b but is not required to} 33. S$9°R = {(a,b) | aisa 
parent of b and b has a sibling}, Ro S = {(a, b) | ais an aunt 


or uncle of b} 35. a) R? b) Ro c) R3 4) R3 0G f)R 
g) Ry h) Ry 37.a) Ry b) R2 ©) R3 d)R?* e) R3 f) R? 
g)R? h)R? 39. b got his or her doctorate under someone 
who got his or her doctorate under a; there is a sequence 
of n + 1 people, starting with a and ending with b, such 
that each is the advisor of the next person in the sequence 
41.a) {(a,b) | a—b = 0,3, 4, 6, , 8, or 9 (mod 12)} 
b) {(a, b) | a = b (mod 12)} c) {(a,b) | a—b = 3,6, 
or 9 (mod 12)}' d) {(a,b) | a —b = 4 o0r8 (mod 12)} 
e) {(a,b) | a—b = 3,4, 6,8, or 9 (mod 12)} 43.8 
45. a) 65,536 b) 32,768 47.a)2"@tD? py angr@—D/2 
c) 3n(n—1)/2 d) gna-l e) gn(n—1)/2 f) gn? =... gnin-l) 
49. There may be no such b. 51. If R is symmetric and 
(a,b) € R, then (b,a) € R, so (a,b) € R—!. Hence, 
RCR!, Similarly, R!CR.SoR=R"!. Conversely, if 
R = R~! and (a,b) € R, then (a, b) € R~!, so (b, a) € R. 
Thus R is symmetric. 53. R is reflexive if and only if 
(a,a) € R foralla € A if and only if (a, a) € R™! [because 
(a, a) € Rif and only if (a, a) € R~']if and only if R7! is re- 
flexive. 55. Use mathematical induction. The result is trivial 
forn = 1. Assume R” is reflexive and transitive. By Theorem 
1, R"t! CR. Tosee that RC R"t! = RoR, let (a, b) € R. 
By the inductive hypothesis, R” = R and hence, is reflexive. 
Thus (b, b) € R”. Therefore (a,b) € R"t!. 57. Use math- 
ematical induction. The result is trivial forn = 1. Assume R” 
is reflexive. Then (a, a) € R” for alla € A and (a,a) € R. 
Thus (a,a) € Ro R = R"*! foralla € A. 59.No, for 
instance, take R = {(1, 2), (2, 1}. 


Section 9.2 


1. {C, 2, 3), d, 2, 4), (1, 3, 4), (2, 3, 4)} 3. (Nadir, 122, 
34, Detroit, 08:10), (Acme, 221, 22, Denver, 08:17), (Acme, 
122,33, Anchorage, 08:22), (Acme, 323, 34, Honolulu 08:30), 
(Nadir, 199, 13, Detroit, 08:47), (Acme, 222, 22, Denver, 
09:10), (Nadir, 322, 34, Detroit, 09:44) 5. Airline and flight 
number, airline and departure time 7. a) Yes b) No c) No 
9. a) Social Security number b) There are no two people with 
the same name who happen to have the same street address. 
c) There are no two people with the same name living together. 
11. (Nadir, 122, 34, Detroit, 08 : 10), (Nadir, 199, 13, Detroit, 
08 : 47), (Nadir, 322, 34, Detroit, 09:44) 13. (Nadir, 122, 
34, Detroit, 08 : 10), (Nadir, 199, 13, Detroit, 08 : 47), (Nadir, 
322, 34, Detroit, 09 : 44), (Acme, 221, 22, Denver, 08 : 17), 
(Acme, 222, 22, Denver,09:10) 15. P3.5.6 


17. Airline | Destination 
Nadir Detroit 
Acme Denver 
Acme | Anchorage 
Acme Honolulu 


19. 
Supplier 


23 
23 
23 
31 
31 
32 
32 
33 


21. Both sides of this equation pick out the subset of R con- 
sisting of those n-tuples satisfying both conditions C; and C2. 
23. Both sides of this equation pick out the set of n-tuples 
that are in R, are in S, and satisfy condition C. 25. Both 
sides of this equation pick out the m-tuples consisting of 
i, th, igth, ..., i,,th components of n-tuples in either R or S. 
27.Let R = {(a,b)} and S = {(a,c)},n = 2,m = 1, 
and i; = 1; P}(R — S) = {(a)}, but P)(R) — Pi(S) = &. 
29. a) Jz followed by P;.3 b) (23, 1), (23, 3), (31, 3), (32, 4) 
31. There is no primary key. 


Section 9.3 
la) [i 1 1] Dlo 1 oO 
& @ <0 1 1 0 
0 0 0 00 1 
of, 1 1] Mfo o 1 
01 1 0 0 0 
00 1 1 0 0 


3. a), 1), C, 3), 2, 2), 3, 1), GB, 3) b) Cd, 2), (2, 2), 
(3,2) oC, 1, Cd, 2), CG, 3), 2, 1), 2, 3), GB, 1), GB, 2), 
(3,3) 5. The relation is irreflexive if and only if the main di- 
agonal of the matrix contains only Os. 7. a) Reflexive, sym- 
metric, transitive b) Antisymmetric, transitive c) Symmetric 
9. a) 4950 b) 9900 c)99 d)100 e)1 11. Change each 0 
toa 1 and each | toa0. 


13.a}0 1 1] DI1 O Of] OF]1 1 1 
1 1 0 0-0 1 1 1 1 
1 1 0 1 0 1 1 1 
iS.a)]0 O 1] BD]1 1 Of] DOJO 1 1 
1 0 0) 1 1 1 1 
Oo 1 1 1 1 1 1 1 1 
17. n*—k 
19. a)! 4 b) ( } 
i 4 
2 3 
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ce) 4 d)! } f 
2 3 2 3 
21. For simplicity we have indicated pairs of edges between 


the same two vertices in opposite directions by using a double 
arrowhead, rather than drawing two separate lines. 


xe 


- {(a, b), (a, c), (b,c), (c, b)} 25. (a, c), (b, a), (c, d), 
e b) 27. {(a, a), (a, b), (a,c), (b, a), (b, b), (b,c), (c, a), 
(c, b), (d, d)} 29. The relation is asymmetric if and only 
if the directed graph has no loops and no closed paths of 
length 2. 31. Exercise 23: irreflexive. Exercise 24: reflex- 
ive, antisymmetric, transitive. Exercise 25: irreflexive, anti- 
symmetric. 33. Reverse the direction on every edge in the 
digraph for R. 35. Proof by mathematical induction. Basis 
step: Trivial for n = 1. Inductive step: Assume true for k. 
Because R§+! = R* o R, its matrix is Mg © Mee. By the 
inductive hypothesis this is Mz © Mi! = M+, 


Section 9.4 
1. a) {(0, 0), (O, 1), C1, 1), C1, 2), (2, 0), (2, 2), (3, 0), (G, 3)} 


b) {(, 1), (, 2), (, 3), (1, 0), CG, D, , 2), (2, 0), 2, D, 
(2,2), (3,0)} 3. {(a, b) | a divides b or b divides a} 


| | 2 ba 


b) a b Cc) a b 


9. a) a 
= c d 
aT. ERS b) 
c d 
c) a b 
| ; | 
13. The symmetric closure of R is R U R7 
Mr V Mp-1 = Merv Mk. 15. Only when R is irreflexive, 


“5 


1 
-Mrur- = 
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in which case it is its own closure. 17. a, a, a, a;a,b,e, a; 
a, d, e, a,b, c, c, b; b, e, a, b; c, b, c, cr 6, ¢, b, c3 cc, ©, ©; d, 
0ad;deedegabeegeadeedegeegedeege, 
ee 19.a) {d, 1), C1, 5), (2, 3), GB, 1D, (3, 2), GB, 3), 3, 4), 
(4, 1), (4,5), 6, 3), 6,4} b){, 1), C, 2), C1, 3), 4), 
(2, 1), (2,5), 3, 1), G, 3), 3,4), 3, 5), 4, D, (4, 2), 4, 3), 
(4, 4), (5, D, G, 3), 6,5)} ©) {d, D, Cd, 3), Cd, 4), Cd, 5), 
(2, 1), (2, 2), (2, 3), 2, 4), 3, 1), (3, 2), G, 3), 3,4), G, 5), 
(4, 1), (4, 3), 4, 4), (4, 5), 6, D, G, 2), G, 3), 6, 4), 5, 5)} 
d) {d, 1), C, 2), C, 3), , 4), 0,5), 2, D, @, 3), 2, 4), 
(2, 5), (3, 1), G, 2), G, 3), 3,4), G, 5), 4, D, (4, 2), (4, 3), 
(4, 4), (4, 5), G, 1), G, 2), (5, 3), 6, 4), (5, 5)} &) {, D, 
(1, 2), C, 3), (1, 4), C, 5), @, 1), (@, 2), 2, 3), (2, 4), @, 5), 
(3, 1), (3, 2), (3, 3), 3.4), 3,5), 4, D, 4, 2), 4, 3), 4,4, 
(4, 5), (5, 1), G, 2), G, 3), 6,4), 6, 5)} 0 {d, D, C, 2), 
(1, 3), , 4), (1, 5), @, D, @, 2), (2, 3), 2, 4), 2,5), GB, D, 
(3, 2), (3, 3), 3, 4), (3,5), 4, 1, 4, 2), 4, 3), 4,4), 4, 5), 
(5, 1), (5, 2), (5, 3), (5, 4), (5, 5)} = 21. a) If there is a stu- 
dent c who shares a class with a and a class with b_ b) If 
there are two students c and d such that a and c share a class, 
c and d share a class, and d and b share a class c) If there 
is a sequence so, ..., 5, of students with n > 1 such that 
SQ = 4,8, = b, and for eachi = 1, 2,...,n, 5; and 
s;-| share aclass 23. The result follows from (R*)7! = 
(Uy Rr")! = LE ae = ea R" = R*. 


25.a)/1 1 1 1] Dlo 0 0 0 
1111 { @ a 4 

tt G4 i @ 2-4 
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6 OO 1 ee a | 
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27. Answers same as for Exercise 25. 29. a) {(1, 1), C1, 2), 
(1, 4), 2, 2), G, 3), 4 D, 4 2), 4 O} bd, D, 
(1, 2), , 4), 2, D, @, 2), 2, 4, G, 3), 4, D, , 2), 
(4,43 oO {d, D, d, 2), d, 4, @, D, @, 2), 2, 4), (3, 3), 
(4, 1), (4, 2), (4,4)} 31. Algorithm 1: O(n38); Algorithm 
2: O(n?) 33. Initialize with A := Me v I, and loop only 
fori := 2ton— 1. 35.a) Because R is reflexive, every 
relation containing it must also be reflexive. b) Both {(0, 0), 
(0, 1), (0, 2), C1, 1), (2, 2)} and {(0, 0), (0, 1), C1, 0), C1, D, 
(2, 2)} contain R and have an odd number of elements, but 
neither is a subset of the other. 


Section 9.5 


1. a) Equivalence relation b) Not reflexive, not transitive 
c) Equivalence relation d) Not transitive e) Not symme- tric, 
not transitive 3. a) Equivalence relation b) Not transitive 
c) Not reflexive, not symmetric, not transitive d) Equivalence 
relation e) Not reflexive, not transitive 5. Many answers 
are possible. (1) Two buildings are equivalent if they were 


opened during the same year; an equivalence class consists of 
the set of buildings opened in a given year (as long as there 
was at least one building opened that year). (2) Two build- 
ings are equivalent if they have the same number of stories; 
the equivalence classes are the set of 1-story buildings, the 
set of 2-story buildings, and so on (one class for each n for 
which there is at least one n-story building). (3) Every build- 
ing in which you have a class is equivalent to every building in 
which you have a class (including itself), and every building 
in which you don’t have a class is equivalent to every building 
in which you don’t have a class (including itself); there are 
two equivalence classes—the set of buildings in which you 
have a class and the set of buildings in which you don’t have a 
class (assuming these are nonempty). 7. The statement “p 
is equivalent to gq" means that p and g have the same entries 
in their truth tables. R is reflexive, because p has the same 
truth table as p. R is symmetric, for if p and q have the same 
truth table, then g and p have the same truth table. If p and g 
have the same entries in their truth tables and q and r have the 
same entries in their truth tables, then p and r also do, so R is 
transitive. The equivalence class of T is the set of all tautolo- 
gies; the equivalence class of F is the set of all contradictions. 
9. a) (x, x) € R because f(x) = f(x). Hence, R is reflexive. 
(x, y) € R if and only if f(x) = f(y), which holds if and 
only if f(y) = f(x) if and only if (y, x) € R. Hence, R is 
symmetric. If (x, y) € R and (y, z) € R, then f(x) = f(y) 
and f(y) = f(z). Hence, f(x) = f(z). Thus, (x, z) € R. 
It follows that R is transitive. b) The sets f —l(b) for b in 
the range of f 11. Let x be a bit string of length 3 or more. 
Because x agrees with itself in the first three bits, (x, x) € R. 
Hence, R is reflexive. Suppose that (x, y) € R. Then x and y 
agree in the first three bits. Hence, y and x agree in the first 
three bits. Thus, (y, x) € R. If (x, y) and (y, z) are in R, then 
x and y agree in the first three bits, as do y and z. Hence, x 
and z agree in the first three bits. Hence, (x, z) € R. It follows 
that Ris transitive. 13. This follows from Exercise 9, where 
f is the function that takes a bit string of length 3 or more to 
the ordered pair with its first bit as the first component and 
the third bit as its second component. 15. For reflexivity, 
((a, b), (a, b)) € R because a+ b = b +a. For symmetry, if 
((a, b), (c,d)) € R,thena+d=b+c,soc+b=d+a, 
so ((c, d), (a, b)) € R. For transitivity, if ((a, b), (c,d)) € R 
and ((c, d), (e, f)) € R,thena+d =b+candc+e=d+f, 
soat+d+c+t+te=b+ct+d+f,soat+e=b+f, 
so ((a, b), (e, f)) € R. An easier solution is to note that by 
algebra, the given condition is the same as the condition that 
f(a, b)) = f ((c, d)), where f ((x, y)) = x — y; therefore by 
Exercise 9 this is an equivalence relation. 17. a) This fol- 
lows from Exercise 9, where the function f from the set of 
differentiable functions (from R to R) to the set of functions 
(from R to R) is the differentiation operator. b) The set of all 
functions of the form g(x) = x2 + C for some constant C 
19. This follows from Exercise 9, where the function f from 
the set of all URLs to the set of all Web pages is the func- 
tion that assigns to each URL the Web page for that URL. 
21.No 23.No 25. R is reflexive because a bit string s 


has the same number of 1s as itself. R is symmetric because 
s and t having the same number of 1s implies that ¢ and s do. 
R is transitive because s and t having the same number of 1s, 
and ¢ and u having the same number of 1s implies that s and 
u have the same number of 1s. 27. a) The sets of people of 
the same age b) The sets of people with the same two parents 
29. The set of all bit strings with exactly two Is. 31. a) The 
set of all bit strings of length 3. b) The set of all bit strings 
of length 4 that end with a 1c) The set of all bit strings of 
length 5 that end 11 d) The set of all bit strings of length 8 
that end 10101 33. Each of the 15 bit strings of length less 
than four is in an equivalence class by itself: [A]r, = {A}, 
(Ole, = (0}, Ha, = {1}, [00]x, = (00}, [O1]e, = {01}, 
. LL11]a, = {111}. The remaining 16 equivalence classes 
are determined by the bit strings of length 4: [0000]r, = 
{0000, 00000, 00001, 000000, 000001, 000010, 000011, 
0000000, ...}, [O001]r, = {0001, 00010, 00011, 000100, 
000101, 000110, 000111, 0001000, ...}, ..., [L11l]R, = 
{1111, 11110, 11111, 111100, 111101, 111110, 111111, 
1111000, ...} 35.a)[2]5 = {i | i=2 (mod5)} = 


{..., -8, —3, 2, 7, 12,...} b) [3]s = {i |i =3 (mod 5)} = 
{...,-7, -2, 3, 8, 13,...} ©) [6]5 = {i | i =6 (mod 5)} = 
{...,-9, -4, 1,6, 11,...} @[-3]ls = fi | i = -3 


(mod 5)} = {..., —8, —3, 2, 7, 12,...} 37. {6n+k|n € Z} 
fork € {0,1,2,3,4,5} 39. a) (C1, 2)] = {(a,b) |a-b= 
—1l} = {d, 2), GB, 4), (4, 5), 6, 6), ...} b) Each equiva- 
lence class can be interpreted as an integer (negative, positive, 
or zero); specifically, [(a, b)] can be interpreted as a — b. 
41.a) Nob) Yesc) Yesd)No 43. (a), (c),(e) 45. (b), (d), 
(e) 47. a) {(0, 0), C1, 1), , 2), (2, 1), (2, 2), GB, 3), 3,4), 
(3, 5), (4, 3), (4, 4), (4,5), GS, 3), 5, 4), 6, 5)} b) {(0, 0), 
(0, 1), G, 0), (, 1), @, 2), @, 3), 3, 2), GB, 3), 4, 4), 
(4,5), (5,4), (5, 5)} ©) {(, 0), , 1), (0, 2), 1, 0), C1, 1), 
(1, 2), (2, 0), 2, 1), @, 2), 3, 3), GB, 4), G, 5), 4, 3), 
(4, 4), (4, 5), (5, 3), (5, 4), (5, 5)} d) {€0, 0), Cl, 1), (2, 2), 
(3, 3), (4, 4), (, 5)} 49.[0]o € [0]3, le ¢ Us, 
[2]e6 © [2]3, [3]6 & [0]s, [4]6 S U1]3, Se S [2]3 S51. Let 
A be a set in the first partition. Pick a particular element x 
of A. The set of all bit strings of length 16 that agree with x on 
the last four bits is one of the sets in the second partition, and 
clearly every stringin Aisinthatset. 53. Weclaim that each 
equivalence class [x]p,, is a subset of the equivalence class 
[x]r,. To show this, choose an arbitrary element y € [x]p,,. 
Then y is equivalent to x under R31, so either y = x or y and x 
are each at least 31 characters long and agree on their first 31 
characters. Because strings that are at least 31 characters long 
and agree on their first 31 characters perforce are at least 8 
characters long and agree on their first 8 characters, we know 
that either y = x or y and x are each at least 8 characters 
long and agree on their first 8 characters. This means that y is 
equivalent to x under Rg, so y € [x]r,. 55. {(a, a), (a, b), 
(a,c), (b, a), (b, b), (b, €), (c, a), (c, B), (c, ©), (d, d), (d, e), 
(e,d), (e,e)} S57.a)Z b){n+ 5 |n € Z} 59. a) Ris 
reflexive because any coloring can be obtained from itself via 
a 360-degree rotation. To see that R is symmetric and tran- 
sitive, use the fact that each rotation is the composition of 
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two reflections and conversely the composition of two reflec- 
tions is a rotation. Hence, (C;, C2) belongs to R if and only 
if Cz can be obtained from C; by a composition of reflec- 
tions. So if (C;, Cz) belongs to R, so does (C2, C,) because 
the inverse of the composition of reflections is also a com- 
position of reflections (in the opposite order). Hence, R is 
symmetric. To see that R is transitive, suppose (C;, C2) and 
(C2, C3) belong to R. Taking the composition of the reflec- 
tions in each case yields a composition of reflections, showing 
that (C,, C3) belongs to R. b) We express colorings with se- 
quences of length four, with r and b denoting red and blue, 
respectively. We list letters denoting the colors of the upper 
left square, upper right square, lower left square, and lower 
right square, in that order. The equivalence classes are: {rrrr}, 
{bbbb}, {rrrb, rrbr, rbrr, brrr}, {bbbr, bbrb, brbb, rbbb}, 
{rbbr, brrb}, {rrbb, brbr, bbrr, rbrb}. 61.5 63. Yes 
65. R67. First form the reflexive closure of R, then form 
the symmetric closure of the reflexive closure, and finally 
form the transitive closure of the symmetric closure of the 
reflexive closure. 69. p(0) = 1, pl) = 1, p@) = 2, 
p(3) = 5, p(4) = 15, p(5) = 52, p(6) = 203, p(7) = 877, 
p(8) = 4140, p(9) = 21147, p10) = 115975 


Section 9.6 


1. a) Isa partial ordering b) Not antisymmetric, not transitive 
c) Is a partial ordering d) Is a partial ordering e) Not anti- 
symmetric, nottransitive 3.a)Nob)Noc) Yes _ 5.a) Yes 
b)No c)Yes d)No 7.a)No_ b)Yes c)No_ 9.No 
11. Yes 13. a) {(0, 0), C1, 0), 1, 1), (2, 0), (2, 1), (2, 2)} 
b)(Z, <) co) (P(Z), 9) d)(Z, “is a multiple of”) 
15. a) {0} and {1}, for instance b)4 and 6, for instance 
17.a) 1, 1,2) < 0,2,1) b) (0,1, 2,3) < (0, 1, 3, 2) 
c) (0, 1, 1, 1,0) < G,0,1,0,1) 19.0 < 0001 < O01 < 
01 < 010 < 0101 < O11 < 11 
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25. (a, b), (a, c), (a, d), (b,c), (b, d), (a, a), (b, b), (c, c), 
(d,d) 27. (a,a), (a, 8), (a, d), (a, e), (a, f), (b, b), (. 8), 
(b, d), (b, €), (b, f), (C0), (c, 8), (€, 4), (c, €)s (c.f), (8, 4), 
(g, €), (8. f)s (8. 8)» (dd), (2). (ff) 29. G, {a}), 
(@, {b}), WD, {c}), Ca}, fa, b}), Cla}, {a, c}), ({b}, {a, 5}), 
({b}, {b, c}), Cc}, {a, ch), Cc}, (2, ch), Ca, b}, {a, b, c}), 
({a, c}, {a, b, c}) ({b, c}, {a, b, c}) 31. Let (S, =) be a fi- 
nite poset. We will show that this poset is the reflexive transi- 
tive closure of its covering relation. Suppose that (a, b) is in 
the reflexive transitive closure of the covering relation. Then 
a = bora ~ b,soa = b, or else there is a sequence 
a, 42,..., dy, Such that a ~ aj ~ az < -:+ ~ ad, < b,in 
which case againa = b by the transitivity of = . Conversely, 
suppose that a < b.If a = b then (a, b) is in the reflex- 
ive transitive closure of the covering relation. If a < b and 
there is no z such that a < z < b, then (a, b) is in the cov- 
ering relation and therefore in its reflexive transitive closure. 
Otherwise, let a ~ aj ~ a2 < -:: <~ a, ~< b bea longest 
possible sequence of this form (which exists because the poset 
is finite). Then no intermediate elements can be inserted, so 
each pair (a, a1), (a1, a2), ..., (An, b) is in the covering re- 
lation, so again (a, b) is in its reflexive transitive closure. 
33. a) 24,45 b)3,5 c)No d)No e)15,45 f) 15 g) 15, 
5,3 h)15 35. a) {1, 2}, {1, 3, 4}, {2, 3, 4} b) {1}, {2}, {4} 
c)No d)No e) {2, 4}, {2,3, 4} f) {2,4} — g) {3, 4}, {4 
h) {3, 4}. 37. Because (a, b) < (a, b), =< is reflexive. If 
(a1, a2) = (bj, bz) and (a1, a2) # (by, bz), either ay ~ by, or 
a, = b, and az ~ bo. Ineither case, (b1, b2) is not less than or 
equal to (a,, a2). Hence, = is antisymmetric. Suppose that 
(a1, a2) < (bj, b2) K~ (c1, c2). Then if ay ~ by or by ~ 1, 
we have a, ~ cy, SO (aj, a2) ~ (c1, C2), but if ay = b) = cc}, 
then a2 ~ bz ~ c2, which implies that (a), a2) ~ (c1, c2). 
Hence, = istransitive. 39. Because (s, ft) < (s, ft), < isre- 
flexive. If (s, t) < (u, v) and (u, v) X (s, ft), thens <u xs 
and t =~ v = t; hence, s = uandt = v. Hence, X is 
antisymmetric. Suppose that (s, tf) < (u, v) = (w, x). Then 
Ss <u,t <v,u =< w, and v = x. It follows that s = w 
and t = x. Hence, (s, tf) <(w, x). Hence, =< is transitive. 
41. a) Suppose that x is maximal and that y is the largest 
element. Then x = y. Because x is not less than y, it fol- 
lows that x = y. By Exercise 40(a) y is unique. Hence, x is 
unique. b) Suppose that x is minimal and that y is the small- 
est element. Then x = y. Because x is not greater than y, it 


follows that x = y. By Exercise 40(b) y is unique. Hence, x is 
unique. 43.a)Yes b)No c)Yes 45. Use mathematical 
induction. Let P (n) be “Every subset with n elements from a 
lattice has a least upper bound and a greatest lower bound.” 
Basis step: P(1) is true because the least upper bound and 
greatest lower bound of {x} are both x. Inductive step: As- 
sume that P(k) is true. Let S be a set with k + | elements. 
Let x € S and S’ = S — {x}. Because S’ has k elements, 
by the inductive hypothesis, it has a least upper bound y and 
a greatest lower bound a. Now because we are in a lattice, 
there are elements z = lub(x, y) andb = glb(x, a). We 
are done if we can show that z is the least upper bound of 
S and b is the greatest lower bound of S. To show that z is 
the least upper bound of S, first note that if w € S, then 
w= xorw € S’.Ifw = x then w W z because z is the 
least upper bound of x and y. If w € S$’, then w = z be- 
cause w = y, which is true because y is the least upper bound 
of S’, and y X z, which is true because z = lub(x, y). To 
see that z is the least upper bound of S, suppose that u is an 
upper bound of S. Note that such an element uw must be an 
upper bound of x and y, but because z = lub(x, y), it follows 
that z < u. We omit the similar argument that b is the great- 
est lower bound of S. 47.a)No_ b) Yes _ ec) (Proprietary, 
{Cheetah, Puma}), (Restricted, {Cheetah, Puma}), (Reg- 
istered, {Cheetah, Puma}), (Proprietary, {Cheetah, Puma, 
Impala}), (Restricted, {Cheetah, Puma, Impala}), (Registered, 
{Cheetah, Puma, Impala}) ) (Non- proprietary, {Impala, 
Puma}), (Proprietary, {Impala, Puma}), (Restricted, {Impala, 
Puma}), (Nonproprietary, {Impala}), (Proprietary, {Impala}), 
(Restricted, {Impala}), (Nonproprietary, {Puma}), (Propri- 
etary, {Puma}), (Restricted, {Puma}), (Nonproprietary, %), 
(Proprietary, %), (Restricted, @) 49. Let TI be the set of all 
partitions of a set S with P; = P 2 if P; is arefinement of Po, 
that is, if every setin P; is a subset of a setin P. First, we show 
that (II, =< ) isa poset. Because P = P for every partition P, 
= is reflexive. Now suppose that P; =< P2 and P) = P. Let 
T € P,. Because P, < Py, there is a set T’ € P» such that 
T CT’. Because P2 = P| there is aset JT” € P; such that 
T’ CT”. Itfollows that T C T”. But because P is a partition, 
T = T”, which implies that T = T’ because T C T’ CT”. 
Thus, T € P. By reversing the roles of P; and P> it follows 
that every set in P> is also in P;. Hence, P} = P2 and = is 
antisymmetric. Next, suppose that P} =< P2 and P2 = P3.Let 
T € P|. Then there is a set T’ € Py such that T C T’. Because 
P) =< P3thereisaset 7” € P3 suchthat T’ C T”. This means 
that T C T”. Hence, P; =< P3. It follows that ~ is transitive. 
The greatest lower bound of the partitions P; and P> is the 
partition P whose subsets are the nonempty sets of the form 
T; 1 T2 where 7; € P; and T2 € P2. We omit the justification 
of this statement here. The least upper bound of the partitions 
P, and P3 is the partition that corresponds to the equivalence 
relation in which x € S is related to y € S if there isa 
sequence x = X09, X1,X2,...,Xn = y for some nonnegative 
integer n such that for eachi from 1 ton, x;_, and x; are in the 
same element of P; or of P>. We omit the details that this is 
an equivalence relation and the details of the proof that this is 


the least upper bound of the two partitions. 51. By Exercise 
45 there is a least upper bound and a greatest lower bound for 
the entire finite lattice. By definition these elements are the 
greatest and least elements, respectively. 53. The least ele- 
ment of a subset of Z* x Z* is that pair that has the smallest 
possible first coordinate, and, if there is more than one such 
pair, that pair among those that has the smallest second co- 
ordinate. 55. If x is an integer in a decreasing sequence of 
elements of this poset, then at most |x| elements can follow x 
in the sequence, namely, integers whose absolute values are 
|x| — 1, |x| — 2, ..., 1, 0. Therefore there can be no infinite 
decreasing sequence. This is not a totally ordered set, because 
5 and —5, forexample, areincomparable. 57. To find which 
of two rational numbers is larger, write them with a positive 
common denominator and compare numerators. To show that 
this set is dense, suppose that x < y are two rational num- 
bers. Then their average, i.e., (x + y)/2, is a rational number 
between them. 59. Let (S, =) be a partially ordered set. It 
is enough to show that every nonempty subset of S contains a 
least element if and only if there is no infinite decreasing se- 
quence of elements a), a2, 43,... in S (i.e., where aj41 ~ aj 
for all 7). An infinite decreasing sequence of elements clearly 
has no least element. Conversely, let A be any nonempty sub- 
set of S that has no least element. Because A is nonempty, 
choose a; € A. Because a, is not the least element of A, 
choose az € A with a2 ~< aj. Because ap is not the least 
element of A, choose a3 € A with a3 ~< a2. Continue in 
this manner, producing an infinite decreasing sequence in S. 
6l.a ~; b <; ¢ ~1 d ~1 @ ~; f ~1 @ ~i h <p i ~; 
Jo ~<t k ~t 1 <; m= 63. 1 ~ 5 ~ 2 ~ 4 ~ 12 ~ 20, 
1 ~ 2 ~5 ~ 4 ~ 12 ~ 20,1 x 2 ~ 4 = 5 ~ 12 ~ 20, 
1<2~4 « 12 <5 ~ 20,1 =~ 5 ~ 2 ~ 4 ~ 20 ~ 12, 
1<«2<5 <4 ~ 20 ~ 12,1~2~<=4<5 <~ 20 < 12 
65. A~Cx ExBx D<x F<G,A<xE*<C<K 
Bx D~<=F<G,C=~AxEx<xBx<xD<F <G, 
Cx E~A<B<D<F<G,E<~A~<C«<B< 
Dx F<«<G,E<xC<~A<xBxD<F<G,A<CK 


BxExD<=F<=G,C~A<xBxEx<D<F<G, 
A<x~C+<x Bx Dx E~x~=F<G,C<A-<B<K 
DxE~=F<=G,A~C<xEx<=xBx<xF<D<G, 
A~ Ex Cw+~B+«~ F«x Dx G,Cw~K~ AWE ~ 
Bx F«=D<G,C<x~ExA~<=Bx<xF<xD<G, 
E<~A~<xCx+<x Bx F< Dx G,ExCe-#«Ax«< 
Bx F«=D<=G,A~Cx Bx Ex=F<D<G, 
C~<~xA~= B< E < F <~ D<G_ 67. Determine user 


needs < Write functional requirements ~< Set up test sites < 
Develop system requirements < Write documentation ~ De- 
velop module A ~< Develop module B ~< Develop module 
C < Integrate modules < a test < 6 test <x Completion 


Supplementary Exercises 


1. a) Irreflexive (we do not include the empty string), symmet- 
ric_b) Irreflexive, symmetric c) Irreflexive, antisymmetric, 
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transitive 3.((a,b), (a,b)) € R becausea+b=a+b. 
Hence, R is reflexive. If ((a, b), (c, d)) € Rthena+d = b+c, 
so that c + b = d +a. It follows that ((c, d), (a, b)) € R. 
Hence, R is symmetric. Suppose that ((a, b), (c, d)) and 
((c, d), (e, f)) belong to R. Thena +d = b+ c and 
c+ f = d +e. Adding these two equations and sub- 
tracting c + d from both sides givesa + f = b+e. 
Hence, ((a, b), (e, f)) belongs to R. Hence, R is transitive. 
5. Suppose that (a, b) € R. Because (b, b) € R it follows that 
(a,b) € R2. 7. Yes, yes 9. Yes, yes 11. Two records 
with identical keys in the projection would have identical keys 
in the original. 13. (AU R)'=A'UR!T=AUR! 
15. a)R = {(a, b), (a, c)}. The transitive closure of the 
symmetric closure of R is {(a, a), (a, b), (a, c), (b, a), 
(b, b), (b,c), (c, a), (c, b), (c, c)} and is different from the 
symmetric closure of the transitive closure of R, which is 
{(a, b), (a, c), (b, a), (c, a)}. b) Suppose that (a, b) is in 
the symmetric closure of the transitive closure of R. We must 
show that (a, b) is in the transitive closure of the symmetric 
closure of R. We know that at least one of (a, b) and (b, a) 
is in the transitive closure of R. Hence, there is either a path 
from a to b in R or a path from b to a in R (or both). In the 
former case, there is a path from a to b in the symmetric clo- 
sure of R. In the latter case, we can form a path from a to b 
in the symmetric closure of R by reversing the directions of 
all the edges in a path from b to a, going backward. Hence, 
(a, b) isin the transitive closure of the symmetric closure of R. 
17. The closure of S with respect to property P is a relation 
with property P that contains R because R C S. Hence, the 
closure of S with respect to property P contains the closure 
of R with respect to property P. 19. Use the basic idea of 
Warshall’s algorithm, except let will equal the length of the 
longest path from vy; to v; using interior vertices with sub- 
scripts not exceeding k, and equal to —1 if there is no such 
path. To find wit from the entries of W;_;, determine for each 
pair (i, 7) whether there are paths from v; to vg, and from vx, 


to vj using no vertices labeled greater than k. If either wit 4 


or rae is —1, then such a pair of paths does not exist, so 


ij = wi 4 If sucha pair of paths exists, then there are 
two possibilities. If wi > 0, there are paths of arbitrary 


long length from v; to v;, so set wil = oo. lf wee = 0, 
set wil — max(wh", wit + week (Initially take 
Wo = Mr.) 21.25 23. Because A; M B; is a subset of 
A; and of B;, the collection of subsets is a refinement of each 
of the given partitions. We must show that it is a partition. By 
construction, each of these sets is nonempty. To see that their 
union is S, suppose that s € S. Because P; and P) are parti- 
tions of S, there are sets A; and B; such thats € A; ands € B;. 
Therefore s € A; 1 B;. Hence, the union of these sets is S. To 
see that they are pairwise disjoint, note that unless i = i’ and 
j= J", (AN Bj)N (AN By) = (Aj NA) (Bj NB;) =. 
25. The subset relation is a partial ordering on any collection 
of sets, because it is reflexive, antisymmetric, and transitive. 


Here the collection of sets is R(S). 27. Find recipe < Buy 
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seafood < Buy groceries < Wash shellfish < Cut ginger and 
garlic < Clean fish < Steam rice < Cut fish < Wash veg- 
etables < Chop water chestnuts < Make garnishes < Cook 
in wok ~ Arrange on platter < Serve 29. a) The only an- 
tichain with more than one element is {c, d}._ b) The only 
antichains with more than one element are {b, c}, {c, e}, and 
{d, e}. c) The only antichains with more than one element 
are {a, b}, {a, c}, {b, c}, {a, b, c}, {d, e}, {d, f}, {e, f}, and 
{d,e, f}. 31. Let (S, =) be a finite poset, and let A be a 
maximal chain. Because (A, =) is also a poset it must have 
a minimal element m. Suppose that m is not minimal in S. 
Then there would be an element a of S with a ~ m. How- 
ever, this would make the set A U {a} a larger chain than A. 
To show this, we must show that a is comparable with every 
element of A. Because m is comparable with every element 
of A and m is minimal, it follows that m ~< x when x is in 
Aandx 4 m. Because a < mandm < x, the transitive 
law shows that a < x for every element of A. 33. Let aRb 
denote that a is a descendant of b. By Exercise 32, if no set 
of n + 1 people none of whom is a descendant of any other 
(an antichain) exists, then k < n, so the set can be partitioned 
into k < n chains. By the pigeonhole principle, at least one of 
these chains contains at leastm-+1 people. 35. We prove by 
contradiction that if $ has no infinite decreasing sequence and 
vx ({VylLy < x > P(y)]} > P(x)), then P(x) is true for all 
x € S§. If it does not hold that P(x) is true for all x € S, let 
x, be an element of S such that P(x) is not true. Then by 
the conditional statement already given, it must be the case 
that VyLy ~ x; — P(y)] is not true. This means that there 
is some x2 with x2 ~< x, such that P(x2) is not true. Again 
invoking the conditional statement, we get an x3 < x2 such 
that P(x3) is not true, and so on forever. This contradicts the 
well-foundedness of our poset. Therefore, P(x) is true for all 
x € S. 37. Suppose that R is a quasi-ordering. Because R 
is reflexive, if a € A, then (a,a) € R. This implies that 
(a,aye€ R—!. Hence, a € RM R7!. It follows that RN R~! is 
reflexive. RM R~! is symmetric for any relation R because, 
for any relation R, if (a,b) € R then (b,a) € R-! and 
vice versa. To show that RM R7! is transitive, suppose that 
(a,b) € ROR! and (b,c) € RN R™!. Because (a,b) € R 
and (b,c) € R, (a,c) € R, because R is transitive. Simi- 
larly, because (a, b) € R—! and (b,c) € R7!,(b,a) E R 
and (c,b) € R, so (c,a) € Rand (a,c) € R~!. Hence, 
(a,c) € RN R7!. It follows that RM Ro! is an equiva- 
lence relation. 39. a) Because glb(x, y) = glb(y, x) and 
lub(x, y) = lub(y, x), it follows that x A y = y A x and 
x Vy = y V x. b) Using the definition, (x A y) A z is 
a lower bound of x, y, and z that is greater than every other 
lower bound. Because x, y, and z play interchangeable roles, 
x A (y A z) is the same element. Similarly, (x V y) Vv z is 
an upper bound of x, y, and z that is less than every other 
upper bound. Because x, y, and z play interchangeable roles, 
x V(yV z) 1s the same element. c) To show thatx A(x Vy) = x 
it is sufficient to show that x is the greatest lower bound of 
x, and x V y. Note that x is a lower bound of x, and be- 
cause x V y is by definition greater than x, x is a lower bound 
for it as well. Therefore, x is a lower bound. But any lower 


bound of x has to be less than x, so x is the greatest lower 
bound. The second statement is the dual of the first; we omit 
its proof. d) x is a lower, and an upper, bound for itself and it- 
self, and the greatest, and least, such bound. 41. a) Because 
1 is the only element greater than or equal to 1, it is the only 
upper bound for | and therefore the only possible value of 
the least upper bound of x and 1. b) Because x = 1, x isa 
lower bound for both x and 1 and no other lower bound can 
be greater than x, sox A 1 = x. c) Because 0 = x, x is 
an upper bound for both x and 0 and no other bound can be 
less than x, sox VO = x. d) Because 0 is the only element 
less than or equal to 0, it is the only lower bound for 0 and 
therefore the only possible value of the greatest lower bound 
of x and0. 43.L = (S,C) where S = {@, {1}, {2}, {3}, 
{1, 2}, {2,3}, (1, 2,3}} 45.Yes 47. The complement of a 
subset X C S is its complement S$ — X. To prove this, note 
that X v (S — X) = land X A (S — X) = O because 
XU(S—X)= Sand X¥N(S— X)=9%. 49. Think of the 
rectangular grid as representing elements in a matrix. Thus we 
number from top to bottom and within that from left to right. 
The partial order is that (a, b) x (c,d) iffa < candb < d. 
Note that (1, 1) is the least element under this relation. The 
rules for Chomp as explained in Chapter | coincide with the 
rules stated in the preamble here. But now we can identify the 
point (a, b) with the natural number p?~!q~! for all a and b 
with 1 < a < mand 1 < b < n. This identifies the points in 
the rectangular grid with the set S in this exercise, and the par- 
tial order < just described is the same as the divides relation, 
because p*—!g?—! | p°—!q4—! if and only if the exponent of 
p on the left does not exceed the exponent of p on the right, 
and similarly for q. 


CHAPTER 10 
Section 10.1 


l.a) 


Boston 


Detroit 
Newark 


Washington 


Miami 


b) 


Boston 
Detroit 


Newark 


Washington 


Miami 


c) Boston 
Detroit 
Newark 
Washington 
Miami 
d) Boston 
Detroit 
Newark 
Washington 
Miami 
e) Boston 
Detroit 
Newark 
Washington 
Miami 
3.Simple graph 5.Pseudograph 7. Directed graph 


9. Directed multigraph 11. If wRv, then there is an edge 
associated with {u, v}. But {u, v} = {v, wu}, so this edge is 
associated with {v, uw} and therefore vRu. Thus, by definition, 
R is a symmetric relation. A simple graph does not allow 
loops; therefore, wu Ru never holds, and so by definition R is 
irreflexive. 


13. a) A, A» 


Ag 
c) Ay Ay 
“ Ln 
As Ag 
15. Hairy 


Hermit thrush Robin woodpecker 


Mockingbird Blue jay Nuthatch 
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17, 
Fermat Descartes 
Mersenne 
| © sn © a ©) 
Aristotle Euclid Eratosthenes 
o e) 
Fibonacci Maurolico 
Oo 
al-Khowarizmi 
. Goldbach 
Lamé 
Bézout 
Stirling 
De Morgan Gauss 
Lovelace Dodgson 
Boole 
19. President 
Director, Director, 
Rand D marketing 
e 
Director, Chief financial 
operations officer 
21. Tigers Blue jays 
Orioles Cardinals 


23. We find the telephone numbers in the call graph for Febru- 
ary that are not present in the call graph for January and vice 
versa. For each number we find, we make a list of the num- 
bers they called or were called by using the edges in the call 
graph. We examine these lists to find new telephone num- 
bers in February that had similar calling patterns to defunct 
telephone numbers in January. 25. We use the graph model 
that has e-mail addresses as vertices and for each message 
sent, an edge from the e-mail address of the sender to the 
e-mail address of the recipient. For each e-mail address, we 
can make a list of other addresses they sent messages to and 
a list of other addresses from which they received messages. 
If two e-mail addresses had almost the same pattern, we con- 
clude that these addresses might have belonged to the same 
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person who had recently changed his or her e-mail address. 
27. Let V be the set of people at the party. Let E be the set 
of ordered pairs (u,v) in V x V such that uv knows v’s name. 
The edges are directed, but multiple edges are not allowed. 
Literally, there is a loop at each vertex, but for simplicity, the 
model could omit the loops. 29. Vertices are the courses; 
edges are directed; edge uv means that course u is prerequisite 
for course v; courses without prerequisites are vertices with 
in-degree 0; courses that are not prerequisite for any other 
courses are vertices with out-degree 0. 31. Let the set of 
vertices be a set of people, and two vertices are joined by an 
edge if the two people were ever married. Ignoring complica- 
tions, this graph has the property that there are two types of 
vertices (men and women), and every edge joins vertices of 
opposite types. 


33. Se 


e 
Sy 

35. Represent people in the group by vertices. Put a directed 
edge into the graph for every pair of vertices. Label the edge 
from the vertex representing A to the vertex representing B 
with a + (plus) if A likes B, a — (minus) if A dislikes B, and 
a 0 if A is neutral about B. 


Section 10.2 


l.v = 6;e = 6; deg(a) = 2, deg(b) = 4, deg(c) = 1, 
deg(d) = 0, deg(e) = 2, deg(f) = 3; c is pendant; d is 
isolated. 3.v = 9;e = 12; deg(a) = 3, deg(b) = 2, 
deg(c) = 4, deg(d) = 0, deg(e) = 6, deg(f) = 0; 
deg(g) = 4; deg(h) = 2; deg(i) = 3; d and f are isolated. 
5.No 7.v = 4,;e = 7; deg (a) = 3, deg (b) = 1, 
deg (c) = 2, deg~(d) = 1, degt(a) = 1, degt(b) = 2, 
degt(c) = 1, degt(d) = 3. 9.5 vertices, 13 edges; 
deg-(a) = 6, deg™(a) = 1, deg™(b) = 1, deg™(b) = 5, 
deg (c) = 2, deg™(c) = 5, deg (d) = 4, deg™(d) = 2, 
deg (e) = 0, degt (e) = 0 


13. The number of coauthors that person has; that per- 
son’s coauthors; a person who has no coauthors; a per- 
son who has only one coauthor 15. In the directed graph 
deg~ (v) = number of calls v received, deg* (v) = number of 


calls v made; in the undirected graph, deg(v) is the number of 
calls either made or received by v. 17. (deg*(v), deg (v)) 
is the win-loss record of v.19. In the undirected graph 
model in which the vertices are people in the group and 
two vertices are adjacent if those two people are friends, 
the degree of a vertex is the number of friends in the 
group that person has. By Exercise 18, there are two ver- 
tices with the same degree, which means that there are two 
people in the group with the same number of friends in 
the group. 21. Bipartite 23. Not bipartite 25. Not bi- 
partite 27. a) Parts {h,s,n,w} and {P, QO, R, S}, E = 
{{P,n},{P,w},{Q, 5}, {Q,n}, {R, n}, {R, w}, {S, h}, {S, s}} 
b) There is. c) {Pw, Qs, Rn, Sh} among others 29. Only 
Barry is willing to marry Umaand Xia. 31. Model this with 
an undirected bipartite graph, with an edge between a man 
and a woman if they are willing to marry each other. By Hall’s 
theorem, it is enough to show that for every set S of women, 
the set N(S) of men willing to marry them has cardinality 
at least |S|. Let m be the number of edges between S and 
N(S). Since every vertex in S has degree k, it follows that 
m = k|S|. Because these edges are incident to N (S), it follows 
that m < k|N(S)|. Therefore k|S| < k|N(S)|, so |N(S)| => 
|S]. 33. a) (fa, b, c, f}, fla, b}, fa, FY, {b, ch, {b, FID 
b) ({a, x, c, f}, {{a, x}, {c, x}, {e, x}}) 35. a)n vertices, 
n(n —1)/2 edges b) n vertices, n edges c) n+ 1 vertices, 2n 
edges d) m-n vertices, mn edges e) 2” vertices, n2”—! edges 
37. a) 3,3, 3,3 b)2,2,2,2 ©)4,3,3,3,3 d)3,3,2,2,2 
e) 3,3, 3,3,3,3,3,3 39. Each of the n vertices is adjacent 
to each of the other n — | vertices, so the degree sequence is 
n—1,n—1,...,n—1(n terms). 


41.7 


43. a) Yes 


b) No c)No d) No 
e) Yes 


e 

f) No. 45. First, suppose that d, d2, ..., dn is graphic. We 
must show that the sequence whose terms are d2 — 1, d3—1.,..., 
dg,41—1, da,4+2, dd, +3,---,n is graphic once it is put into non- 
increasing order. In Exercise 44 it is proved that if the original 
sequence is graphic, then in fact there is a graph having this 
degree sequence in which the vertex of degree d; is adjacent 
to the vertices of degrees d>, d3,..., dq, +1. Remove from this 
graph the vertex of highest degree (d;). The resulting graph 
has the desired degree sequence. Conversely, suppose that d1, 
dp,..., dy 1S a nonincreasing sequence such that the sequence 


dy = 1, a3 = i Pare day4i = 1, dqy,+2, dy 4300+ dy is graphic 
once it is put into nonincreasing order. Take a graph with this 
latter degree sequence, where vertex v; has degree d; — 1 for 
2 <i <d,+1 and vertex v; has degree d; for dj +2 <i <n. 
Adjoin one new vertex (call it v;), and put in an edge from v1 
to each of the vertices v2, v3,..., Va,41. The resulting graph 
has degree sequence d}, d2,...,dn. 47. Let di, do,...,dn bea 
nonincreasing sequence of nonnegative integers with an even 
sum. Construct a graph as follows: Take vertices v1, v2,..., Vp 
and put |d;/2| loops at vertex v;, fori = 1, 2,...,m. For 
each i, vertex v; now has degree either d; or dj; — 1. Because 
the original sum was even, the number of vertices for which 
deg(v;) = dj; — 1 is even. Pair them up arbitrarily, and put in 
an edge joining the vertices in each pair. 49. 17 


51.Ja blag eo” ine a b 
] ; c® ®i\c d|\c® d 
de eo” dQ eo ae de e” 
c® @i\c® @7|c® d|\c® ej 
a b de eo de o” de eo” 
] c® c® c® 
a b de eo” av de eo 
: ey d ej 


Dik ae 


c® ey 


53. a) For alln > 1 b)Foralln > 3 c)Forn =3 d) For 
alln >O 55.5 


57a f b 


59. a) The graph with n vertices and no edges _b) The disjoint 
union of K,, and K,, ¢) The graph with vertices {v,,..., vn} 
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with an edge between v; and v; unlessi = j + | (mod n) 
d) The graph whose vertices are represented by bit strings of 
length n with an edge between two vertices if the associated 
bit strings differ in more than one bit 61. v(v — 1)/2 —e 
63.n-—1—d,,n-—1—d,_-},..,.n —-l-d,n-1—-—d 
65. The union of G and G contains an edge between each pair 
of the n vertices. Hence, this union is Ky. 

67. Exercise 7: 


Exercise 8: 


69. A directed graph G = (V, E) is its own converse if and 
only if it satisfies the condition (u,v) € E if and only if 
(v, u) € E. But this is precisely the condition that the associ- 
ated relation must satisfy to be symmetric. 


71. P(O,0) P(O,1) P(O, 2) 


Pd, 0) 


P(1, 1) | Pd, 2) 


73. We can connect P(i, j) and P(k, /) by using |i — k| hops 
to connect P(i, j) and P(k, j) and |j — /| hops to connect 
P(k, j) and P(k, 1). Hence, the total number of hops required 
to connect P(i, 7) and P(k, 1) does not exceed |i —k|+|j—/]. 
This is less than or equal tom + m = 2m, which is O(m). 


Section 10.3 
1. Adjacent 3. Terminal 
Vertex | Vertices Vertex | Vertices 
a b,c,d a a,b,c,d 
b a,d b d 
Cc a,d c a,b 
d a,b,c d b,c,d 
5.J/0 1 1 1 
1 0 0 1 
1 0 0 1 
1 1 1 0 
7 [1 £ 1 1] %aMlo 1 1 1 
0 0 0 1 1 0 1 1 
1 1 0 0 i: i. Ov 4 
Oo 1 1 1 1 1 1 +0 
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b)fo 1 1 1 11 Of0 0 1 1 1 
1 0 0 0 0 0 0 1 1 41 
1 0 0 0 0 1 1 0 0 0 
1 0 0 0 0 1 1 0 0 0 
1 0 0 0 0 1 1 0 0 0 
MYfo 1 0 1] %fo 101 1 
1 0 1 0 1 01 0 1 
0 1 0 1 0 1 0 1 #1 
1 0 1 0 1 01 0 1 
1 1 1 1 +0 
Nfo 1 10 10 0 0 
100 1 0 1 0 0 
1 00 1 0 0 1 +0 
0 1 1 00 0 0 1 
100 00 1 1 +0 
0 1 00 1 0 0 41 
0 0 1 0 10 0 41 
0 0 0 1 0 1 1 «0 
ll. a b io 9 -% © 
0 0 1 2 
1 1 0 1 
0. 2 1 0 
15.7; 9 2 1 17. ( )a ua 
0 1 1 2 
2 1 41 +0 
1 2-0 17 
d c 
19%.7o 1 0 oO] 21-71 1 2 1 
0 1 #1 =+0 1 0 0 2 
Oo 1 1 «21 1 01 1 
1 0 0 0 0 2 1 ~0 
23. 25. Yes 
as ) 
27. Exercise 13: 1 00 0 0 
0 1 1 1 ~=«0 
1 1 0 0 1 
0 0 1 1 #1 
Exercise 14: 111100 0 0 
1 1 1 0 1 0 0 0 
0 00 01 1 1 «21 
0 00 10 1 1 «21 
Exercise 15:[); 1 1 1 0 0 00 0 0 
0 0 0 0 1 1 21 «1 0 0 
0 1 1 0 0 1 0 0 1 =0 
000 1001 10 1 


29. deg(v) — number of loops at v; deg~(v) 31. 2 if e is not 
a loop, | if e is a loop 


33.a)f1 1 --. 1 0 0 
| Ose, O 4 0 
0 1 0 1 0 
0 0 0 0 1 
i o 1 0 1 
bit O os ® 4 
f- 4 626 0 o 
0 1 0 0 
0 0 1 
0 0 4 
O10 OG 2 6 1 4 1 
1 0 0 
B 1 0 
O Oise J 


where B is the answer to (b) 


Mii 1 --. 1 0 0 
0 -- 0 1 0 
0 O 0) 1 
1 O 0 1 0 
0 1 0 1 
0. 0 228 ZT 10 wes <0 


35. Isomorphic 37.Isomorphic 39. Isomorphic 41. Not 
isomorphic 43.Isomorphic 45. G is isomorphic to itself 
by the identity function, so isomorphism is reflexive. Suppose 
that G is isomorphic to H. Then there exists a one-to-one 
correspondence f from G to H that preserves adjacency and 
nonadjacency. It follows that f—! is a one-to-one correspon- 
dence from H to G that preserves adjacency and nonadja- 
cency. Hence, isomorphism is symmetric. If G is isomorphic 
to H and H is isomorphic to K, then there are one-to-one 
correspondences f and g from G to H and from H to K that 
preserve adjacency and nonadjacency. It follows that go f is 
a one-to-one correspondence from G to K that preserves ad- 
jacency and nonadjacency. Hence, isomorphism is transitive. 
47. Allzeros 49. Label the vertices in order so that all of the 
vertices in the first set of the partition of the vertex set come 
first. Because no edges join vertices in the same set of the par- 
tition, the matrix has the desired form. 51.C5 53.n=5 
only 55.4 57.a)Yes b)No c)No 59.G=(Vj, Fi) 
is isomorphic to H = (V2, E2) if and only if there exist func- 
tions f from V; to V2 and g from E| to E> such that each is 
a one-to-one correspondence and for every edge e in FE) the 
endpoints of g(e) are f(v) and f(w) where v and w are the 


endpoints of e. 61.Yes 63.Yes 65. If f is an isomor- 
phism from a directed graph G to a directed graph H, then f 
is also an isomorphism from G°°”” to H°°"”. To see this note 
that (u, v) is an edge of G°°”” if and only if (v, u) is an edge 
of G if and only if (f(v), f(u)) is an edge of H if and only 
if (f(u), f(v)) is an edge of H°°"”. 67. Many answers are 
possible; for example, Ce and C3 UC3. 69. The product is 
[aij] where aj; is the number of edges from vj to vj when 
i # j and aj; is the number of edges incident tov;. 71. The 
graphs in Exercise 41 provide a devil’s pair. 


Section 10.4 


1. a) Path of length 4; not a circuit; not simple b) Not a path 
c) Nota path d) Simple circuit of length5 3.No  5.No 
7. Maximal sets of people with the property that for any two 
of them, we can find a string of acquaintances that takes us 
from one to the other 9%. If a person has Erdés number n, 
then there is a path of length n from that person to Erdés in 
the collaboration graph, so by definition, that means that that 
person is in the same component as Erdés. If a person is in the 
same component as Erdés, then there is a path from that person 
to Erdés, and the length of the shortest such path is that per- 
son’s Erdés number. 11. a) Weakly connected b) Weakly 
connected c) Not strongly or weakly connected 13. The 
maximal sets of phone numbers for which it is possible to 
find directed paths between every two different numbers in 
the set 15. a) {a,b, f}, {c,d,e} b) {a, b,c, d,e, h}, {f}, 
{g} co) {a, b, d,e, f, g,h, i}, {c} 17. Suppose the strong 
components of wu and v are not disjoint, say with vertex w in 
both. Suppose x is a vertex in the strong component of u. 
Then x is also in the strong component of v, because there 
is a path from x to v (namely the path from x to u fol- 
lowed by the path from u to w followed by the path from 
w to v) and vice versa. Thus x is in the strong component 
of v. This shows that the strong component of u is a sub- 
graph of the strong component of v, and equality follows by 
symmetry. 19.a)2 b)7 c)20 d)61 21. Not isomor- 
phic (G has a triangle; H does not) 23. Isomorphic (the 
path w1, v2, U7, U6, Us, U4, U3, Ug, Uy Corresponds to the path 
V1, V2, V3, V4, V5, V8,V7,¥6,V1) 25.a)3 b)O c)27 d)O 
27.a)1 b)O c)2 d)1 e)5 f)3 29. R is reflexive by 
definition. Assume that (u, v) € R; then there is a path from 
u to v. Then (v, uv) € R because there is a path from v to 
u, namely, the path from wu to v traversed backward. Assume 
that (u, v) € R and (v, w) € R; then there are paths from u 
to v and from v to w. Putting these two paths together gives 
a path from u to w. Hence, (u, w) € R. It follows that R is 
transitive. 31.c 33.b,c,e,i 35. Ifa vertex is pendant 
it is clearly not a cut vertex. So an endpoint of a cut edge that 
is a cut vertex is not pendant. Removal of a cut edge produces 
a graph with more connected components than in the original 
graph. If an endpoint of a cut edge is not pendant, the con- 
nected component it is in after the removal of the cut edge 
contains more than just this vertex. Consequently, removal of 
that vertex and all edges incident to it, including the original 
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cut edge, produces a graph with more connected components 
than were in the original graph. Hence, an endpoint of a cut 
edge that is not pendant is a cut vertex. 37. Assume there 
exists a connected graph G with at most one vertex that is not 
a cut vertex. Define the distance between the vertices u and 
v, denoted by d(u, v), to be the length of the shortest path 
between u and v in G. Let s and ¢ be vertices in G such that 
d(s, t) is a maximum. Either s or f (or both) is a cut vertex, 
so without loss of generality suppose that s is a cut vertex. 
Let w belong to the connected component that does not con- 
tain t of the graph obtained by deleting s and all edges inci- 
dent to s from G. Because every path from w to f contains s, 
d(w,t) > d(s, t), which is acontradiction. 39. a) Denver— 
Chicago, Boston—New York b) Seattle—Portland, Portland— 
San Francisco, Salt Lake City-Denver, New York—Boston, 
Boston—Burlington, Boston—Bangor 41.A minimal set of 
people who collectively influence everyone (directly or indi- 
rectly); {Deborah} 43. An edge cannot connect two vertices 
in different connected components. Because there are at most 
C(n;, 2) edges in the connected component with n; vertices, 
it follows that there are at most an C(nj;, 2) edges in the 
graph. 45. Suppose that G is not connected. Then it has a 
component of k vertices for some k, 1 < k < n— 1. 
The most edges G could have is C(k, 2) + C(n — k, 2) = 
[kk -l)+(n—kK(n—k—1))/2 =k —nk + (n? —n)/2. 
This quadratic function of f is minimized atk = n/2 and 
maximized atk = 1 ork = n — 1. Hence, if G is not 
connected, the number of edges does not exceed the value of 
this function at 1 and at n — 1, namely, (n — 1)(n — 2)/2. 
47.a)1 b)2 c)6 d)21 49. a) Removing an edge from a 
cycle leaves a path, which is still connected. b) Removing an 
edge from the cycle portion of the wheel leaves that portion 
still connected and the central vertex still connected to it as 
well. Removing a spoke leaves the cycle intact and the central 
vertex still connected to it as well. c) Any four vertices, two 
from each part of the bipartition, are connected by a 4-cycle; 
removing one edge does not disconnect them. d) Deleting 
the edge joining (bj, bo, ... , bi-1, 0, bi4i, ... , bn) 
and (bj, bo, ... , bi-1, 1, bi41, ... , bn) does not dis- 
connect the graph because these two vertices are still 
joined via the path (bj, bo, ... , bj-1, 0, bi41, ... , 0), 
(bi, bo, ... , bi-1, 0, Bian, ..., LD, (hi, bo, ..., bi-1, 1, 
bist, or 1), (b,, bo, ea bi-1, Ai. bi+t, ang Od ifn < 2 
and b, = 0, and similarly in the other three cases. 51. If 
G is complete, then removing vertices one by one leaves a 
complete graph at each step, so we never get a disconnected 
graph. Conversely, if edge uv is missing from G, then remov- 
ing all the vertices except uv and v creates a disconnected graph. 
53. Both equal min(@m,n). 55. Let G be a graph with n ver- 
tices; then« (G) < n—1.Let C be asmallest edge cut, leaving 
a nonempty proper subset S of the vertices of G disconnected 
from the complementary set S’ = V — S. If xy is an edge of G 
for every x € Sand y € S’, then the size of C is | S||.S’|, which 
is at leastn — 1, sox(G) < A(G). Otherwise, let x € S and 
y € S’ be nonadjacent vertices. Let T consist of all neighbors 
of x in S’ together with all vertices of S — {x} with neighbors 
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in S’. Then T is a vertex cut, because it separates x and y. 
Now look at the edges from x to T  S’ and one edge from 
each vertex of TM S to S’; this gives us |7| distinct edges that 
lie in C, so A(G) = |C| = |T| > K(G). 57.2 59. Let 
the simple paths P; and P2 be u = x0, X1,...,%n = v and 
u = yo, Y1,---; Ym = V, respectively. The paths thus start out 
at the same vertex. Since the paths do not contain the same set 
of edges, they must diverge eventually. If they diverge only 
after one of them has ended, then the rest of the other path is 
a simple circuit from v to v. Otherwise we can suppose that 
XO = yO. X1 = Vi, .--, Xi = yi, but xj41 A yi41. To form our 
simple circuit, we follow the path y;, yj+1, yj+2, and so on, 
until it once again first encounters a vertex on P; (possibly as 
early as y;+1, no later than y,,). Once we are back on P;, we 
follow it along—forwards or backwards, as necessary—to re- 
turn to x;. Since x; = yj, this certainly forms a circuit. It must 
be a simple circuit, since no edge among the x,s or the ys 
can be repeated (P; and P> are simple by hypothesis) and no 
edge among the x;,s can equal one of the edges y; that we used, 
since we abandoned P» for P; as soonas we hit P;. 61. The 
graph G is connected if and only if every off-diagonal entry of 
A+A’?+A3+4..-.+A"~ is positive, where A is the adjacency 
matrix of G. 63. If the graph is bipartite, say with parts A 
and B, then the vertices in every path must alternately lie in 
A and B. Therefore a path that starts in A, say, will end in B 
after an odd number of steps and in A after an even number of 
steps. Because a circuit ends at the same vertex where it starts, 
the length must be even. Conversely, suppose that all circuits 
have even length; we must show that the graph is bipartite. 
We can assume that the graph is connected, because if it is 
not, then we can just work on one component at a time. Let 
v be a vertex of the graph, and let A be the set of all vertices 
to which there is a path of odd length starting at v, and let 
B be the set of all vertices to which there is a path of even 
length starting at v. Because the component is connected, ev- 
ery vertex lies in A or B. No vertex can lie in both A and B, 
because if one did, then following the odd-length path from v 
to that vertex and then back along the even-length path from 
that vertex to v would produce an odd circuit, contrary to the 
hypothesis. Thus, the set of vertices has been partitioned into 
two sets. To show that every edge has endpoints in different 
parts, suppose that xy is an edge, where x € A. Then the 
odd-length path from v to x followed by xy produces an even- 
length path from v to y, so y € B. (Similarly, if x € B.) 
65. (A, W, H2W2(boat), 0) — (H2W2, H,W,(boat)) > 
(A, H2W2(boat), Wy) —- (Wo, Hy W, A2(boat)) > 
(H2 W2(boat), H,W1) > (0, HW) W, Ho W2 (boat) ) 


Section 10.5 


1. Neither 3. No Euler circuit; a, e, c, e, b, e, d,b, a, c,d 
5.a,b,¢c,4,c,¢4,beaeda T.aqihgdefycagehd 
6 a,b,i,c,b,ha 9%.No,Astillhasodddegree. 11. When 
the graph in which vertices represent intersections and edges 


streets has an Euler path 13.Yes 15.No_ 17. If there is 
an Euler path, then as we follow it each vertex except the 
starting and ending vertices must have equal in-degree and 
out-degree, because whenever we come to a vertex along an 
edge, we leave it along another edge. The starting vertex must 
have out-degree | larger than its in-degree, because we use 
one edge leading out of this vertex and whenever we visit 
it again we use one edge leading into it and one leaving it. 
Similarly, the ending vertex must have in-degree 1 greater 
than its out-degree. Because the Euler path with directions 
erased produces a path between any two vertices, in the un- 
derlying undirected graph, the graph is weakly connected. 
Conversely, suppose the graph meets the degree conditions 
stated. If we add one more edge from the vertex of deficient 
out-degree to the vertex of deficient in-degree, then the graph 
has every vertex with equal in-degree and out-degree. Be- 
cause the graph is still weakly connected, by Exercise 16 this 
new graph has an Euler circuit. Now delete the added edge to 
obtain the Euler path. 19. Neither 21. No Euler circuit; 
a,d,e,d,b,a,e,c,e,b,c,b,e 23.Neither 25. Follow 
the same procedure as Algorithm 1, taking care to follow the 
directions of edges. 27. a)n = 2 b)None_ c) None 
d)n = 1. 29. Exercise 1:1 time; Exercises 2—7: 0 times 
31. a, b, c, d, e, ais a Hamilton circuit. 33. No Hamilton 
circuit exists, because once a purported circuit has reached e 
it would have nowhere to go. 35. No Hamilton circuit ex- 
ists, because every edge in the graph is incident to a vertex of 
degree 2 and therefore must be in the circuit. 37.aqb6f, 
d,eisaHamilton path. 39. f e, d, a, b, cis a Hamilton path. 
41. No Hamilton path exists. There are eight vertices of degree 
2, and only two of them can be end vertices of a path. For each 
of the other six, their two incident edges must be in the path. It 
is not hard to see that if there is to be a Hamilton path, exactly 
one of the inside corner vertices must be an end, and that this 
is impossible. 43.4, b, c, f i, h, g, d, e is a Hamilton path. 
45.m =n >2_ 47. a) (i) No, (ii) No, (iii) Yes b) (4) No, 
(ii) No, (iil) Yes c) (i) Yes, (11) Yes, (111) Yes d) (i) Yes, (ii) Yes, 
(iii) Yes 49. The result is trivial for n = 1: code is 0, 1. As- 
sume we have a Gray code of order n. Let cj, ..., Cx, k = 2” 
be such a code. Then 0c), ..., Ocx, lex, ..., ley is a Gray 
code of order n + 1. 


51. procedure Fleury(G = (V, E): connected multigraph 
with the degrees of all vertices even, V = {v1,..., Vn}) 
vi= vy 
circuit ‘= v 
H:=G 
while H has edges 
e := first edge with endpoint v in H (with respect 
to listing of V) such that e is not a cut edge of H, if 
one exists, and simply the first edge in H with 
endpoint v otherwise 
w := other endpoint of e 
circuit := circuit with e, w added 
vi=w 
H:=H-—-e 
return circuit {circuit is an Euler circuit} 


53. If G has an Euler circuit, then it also has an Euler path. 
If not, add an edge between the two vertices of odd degree 
and apply the algorithm to get an Euler circuit. Then delete 
the new edge. 55. Suppose G = (V, E) is a bipartite graph 
with V = V,; U V2, where V; NM V2 = J and no edge 
connects a vertex in V; and a vertex in V2. Suppose that G 
has a Hamilton circuit. Such a circuit must be of the form 
a1, b1, do, bo, ..., ag, be, ay, where aj € V; and bj € V2 
fori = 1,2,...,k. Because the Hamilton circuit visits each 
vertex exactly once, except for v;, where it begins and ends, 
the number of vertices in the graph equals 2k, an even num- 
ber. Hence, a bipartite graph with an odd number of vertices 
cannot have a Hamilton circuit. 


57.1 2 3 4 


9 10 11 12 


A knight’s tour can be made by following the moves 8, 10, 1, 
7,9,2,11,5,3,12,6,4. 61. We represent the squares of a 
4 x 4 chessboard as follows: 


DCC 


There are only two moves from each of the four corner squares. 
If we include all the edges 1-10, 1-7, 16-10, and 16-7, a cir- 
cuit is completed too soon, so at least one of these edges must 
be missing. Without loss of generality, assume the path starts 
1-10, 10-16, 16-7. Now the only moves from square 3 are 
to squares 5, 10, and 12, and square 10 already has two inci- 
dent edges. Therefore, 3-5 and 3—12 must be in the Hamilton 
circuit. Similarly, edges 8—2 and 8-15 must be in the circuit. 
Now the only moves from square 9 are to squares 2, 7, and 
15. If there were edges from square 9 to both squares 2 and 
15, a circuit would be completed too soon. Therefore the edge 
9-7 must be in the circuit giving square 7 its full complement 
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of edges. But now square 14 is forced to be joined to squares 
5 and 12, completing a circuit too soon (S—14—12-3-5). This 
contradiction shows that there is no knight’s tour on the 4 x 4 
board. 63. Because there are mn squares on anm xn board, 
if both m and n are odd, there are an odd number of squares. 
Because by Exercise 62 the corresponding graph is bipartite, 
by Exercise 55 it has no Hamilton circuit. Hence, there is no 
reentrant knight’s tour. 65. a) If G does not have a Hamil- 
ton circuit, continue as long as possible adding missing edges 
one at a time in such a way that we do not obtain a graph 
with a Hamilton circuit. This cannot go on forever, because 
once we’ve formed the complete graph by adding all miss- 
ing edges, there is a Hamilton circuit. Whenever the process 
stops, we have obtained a (necessarily noncomplete) graph H 
with the desired property. b) Add one more edge to H. This 
produces a Hamilton circuit, which uses the added edge. The 
path consisting of this circuit with the added edge omitted is 
a Hamilton path in H. c) Clearly v; and v, are not adjacent 
in H, because H has no Hamilton circuit. Therefore they are 
not adjacent in G. But the hypothesis was that the sum of 
the degrees of vertices not adjacent in G was at least n. This 
inequality can be rewritten as n — deg(v,) < deg(v,). But 
n — deg(v,) is just the number of vertices not adjacent to v,. 
d) Because there is no vertex following v, in the Hamilton 
path, v, is not in S. Each one of the deg(v1) vertices adjacent 
to vy gives rise to an element of S, so S contains deg(v ) ver- 
tices. e) By part (c) there are at most deg(v,) — 1 vertices other 
than v, not adjacent to v,, and by part (d) there are deg(v1) 
vertices in S, none of which is v,,. Therefore at least one vertex 
of S is adjacent to v,,. By definition, if vg is this vertex, then H 
contains edges vyv, and vj vg41, Where 1 < k <n—1. f) Now 
V1,V2,+++5 Vk—-1, Ves Vay Vn—1, +++» Ve+1, Vi is a Hamilton cir- 
cuit in H, contradicting the construction of H. Therefore, our 
assumption that G did not originally have a Hamilton circuit 
is wrong, and our proof by contradiction is complete. 


Section 10.6 


1. a) Vertices are the stops, edges join adjacent stops, weights 
are the times required to travel between adjacent stops. 
b) Same as part (a), except weights are distances between ad- 
jacent stops. c) Same as part (a), except weights are fares 
between stops. 3.16 5. Exercise 2: a, b, e, d, z;  Ex- 
ercise 3: a, c, d, e, g, z; Exercise 4: a, b, e, h, Lm, p, s, Z 
Taagdbagdfocddfeb,deg,z 9.a) Direct 
b) Via New York c) Via Atlanta and Chicago d) Via New York 
11. a) Via Chicago b) Via Chicago c) ViaLos Angeles d) Via 
Chicago 13. a) ViaChicago b) Via Chicago c) Via Los An- 
geles d) Via Chicago 15. Do not stop the algorithm when 
z is added to the set S. 17. a) Via Woodbridge, via Wood- 
bridge and Camden b) Via Woodbridge, via Woodbridge and 
Camden 19. For instance, sightseeing tours, street cleaning 
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23. O(n?) 25. a-c—-b-d-a (or the same circuit starting at 
some other point and/or traversing the vertices in reverse 
order) 27. San Francisco—Denver—Detroit-New York—Los 
Angeles—San Francisco (or the same circuit starting at some 
other point and/or traversing the vertices in reverse order) 
29. Consider this graph: 


100 


The circuit a-b-a-c-a visits each vertex at least once (and the 
vertex a twice) and has total weight 6. Every Hamilton circuit 
has total weight 103. 31. Let v1, v2,..., v» be a topological 
ordering of the vertices of the given directed acyclic graph. Let 
w(i, j) be the weight of edge v;v;. Iteratively define P(i) with 
the intent that it will be the weight of a longest path ending at 
y; and C(i) with the intent that it will be the vertex preceding 
vy; in some longest path: For i from 1 to n, let P(z) be the 
maximum of P(j) + w(j, i) over all j < i such that vj1j is 
an edge in the directed graph (and if such a j exists let C(i) 
be a value of j for which this maximum is achieved) and let 
P(i) = O if there are no such values of 7. At the conclusion 
of this loop, a longest path can be found by choosing i that 
maximizes P (i) and following the C links back to the start of 
the path. 


Section 10.7 
1. Yes 3. b 5. No 
a d 
c 
7. Yes a d 


b 


9.No_ 11.A triangle is formed by the planar representation 
of the subgraph of Ks consisting of the edges connecting v1, 
v2, and v3. The vertex v4 must be placed either within the tri- 
angle or outside of it. We will consider only the case when 
v4 is inside the triangle; the other case is similar. Drawing the 


three edges from v,, v2, and v3 to v4 forms four regions. No 
matter which of these four regions vs is in, it is possible to 
join it to only three, and not all four, of the other vertices. 
13.8 15. Because there are no loops or multiple edges and 
no simple circuits of length 3, and the degree of the unbounded 
region is at least 4, each region has degree at least 4. Thus 
2e > 4r,orr < e/2. Butr = e —v + 2, so we have 
e—v+2 < e/2, which implies thate < 2v—4. 17.Asin 
the argument in the proof of Corollary 1, we have 2e > 5r and 
r =e—v+2. Thus e —v+2 < 2e/5, which implies that e < 
(5/3)v—(10/3). 19. Only(a)and(c) 21. Nothomeomor- 
phic to K33  23.Planar 25.Nonplanar 27.a)1 b)3 
c)9 d)2 e)4 f)16 29. Draw K;,,, as described in the 
hint. The number of crossings is four times the number in 
the first quadrant. The vertices on the x-axis to the right of 
the origin are (1, 0), (2, 0), ..., (@m/2, 0) and the vertices on 
the y-axis above the origin are (0, 1), (0, 2), ..., (0, 2/2). 
We obtain all crossings by choosing any two numbers a and 
b with 1 < a < b < m/2 and two numbers r and s 
with 1 < r < s < n/2; we get exactly one crossing 
in the graph between the edge connecting (a, 0) and (0, s) 
and the edge connecting (b, 0) and (0, 7). Hence, the number 
of crossings in the first quadrant is C (4,2) - C (5,2) = 
(mf nl 2-1), @/ Se) 2-) Hence, the total number of cross- 
ings is 4- mn(m — 2)(n — 2)/64 = mn(m — 2)(n — 2)/16. 
3l.a)2 b)2 c)2 d)2 e)2 f)2 33. The formula is valid 
forn < 4.Ifn > 4, by Exercise 32 the thickness of K, is 
at least C(n, 2)/(n — 6) = (n+14 ,)/6 rounded up. 
Because this quantity is never an integer, it equals |(n+7)/6]. 
35. This follows from Exercise 34 because Ky,» has mn edges 
and m+n vertices and has no triangles because it is bipartite. 


37. 


Section 10.8 


1. Four colors 3. Three colors 
B A 


wo. eA 


D 


5.3. 7.3 9.2 11.3  13.Graphswithnoedges 15.3 
ifniseven,4ifnisodd 17. Period 1: Math 115, Math 185; 
period 2: Math 116, CS 473; period 3: Math 195, CS 101; 
period 4: CS 102; period 5: CS 273 19.5 21. Exercise 5: 
3 Exercise 6:6 Exercise 7:3 Exercise 8:4 Exercise 9: 3 
Exercise 10:6 Exercise 11:4 23. a) 2 ifn is even, 3 if n is 
odd b)n 25. Two edges that have the same color share no 
endpoints. Therefore if more than n/2 edges were colored the 


same, the graph would have more than 2(n/2) = n vertices. 
27.5 29. Color 1: e, f, d; color 2: c, a, i, g; color 3: h, b, j 
31. Color Cg 33. Four colors are needed to color W,, when 
n is an odd integer greater than 1, because three colors are 
needed for the rim (see Example 4), and the center vertex, be- 
ing adjacent to all the rim vertices, will require a fourth color. 
To see that the graph obtained from W,, by deleting one edge 
can be colored with three colors, consider two cases. If we 
remove a rim edge, then we can color the rim with two colors, 
by starting at an endpoint of the removed edge and using the 
colors alternately around the portion of the rim that remains. 
The third color is then assigned to the center vertex. If we 
remove a spoke edge, then we can color the rim by assigning 
color #1 to the rim endpoint of the removed edge and colors 
#2 and #3 alternately to the remaining vertices on the rim, 
and then assign color #1 to the center. 35. Suppose that G 
is chromatically k-critical but has a vertex v of degree k — 2 
or less. Remove from G one of the edges incident to v. By 
definition of “k-critical,” the resulting graph can be colored 
with k — 1 colors. Now restore the missing edge and use this 
coloring for all vertices except v. Because we had a proper 
coloring of the smaller graph, no two adjacent vertices have 
the same color. Furthermore, v has at most k — 2 neighbors, so 
we can color v with an unused color to obtain a proper (k — 1)- 
coloring of G. This contradicts the fact that G has chromatic 
number k. Therefore, our assumption was wrong, and every 
vertex of G must have degree at leastk—1. 37.a)6 b)7 
c)9 d)11l 39. Represent frequencies by colors and zones 
by vertices. Join two vertices with an edge if the zones these 
vertices represent interfere with one another. Then a k-tuple 
coloring is precisely an assignment of frequencies that avoids 
interference. 41. We use induction on the number of ver- 
tices of the graph. Every graph with five or fewer vertices can 
be colored with five or fewer colors, because each vertex can 
get a different color. That takes care of the basis case(s). So 
we assume that all graphs with k vertices can be 5-colored 
and consider a graph G with k + | vertices. By Corollary 2 in 
Section 10.7, G has a vertex v with degree at most 5. Remove 
v to form the graph G’. Because G’ has only k vertices, we 
5-color it by the inductive hypothesis. If the neighbors of v 
do not use all five colors, then we can 5-color G by assigning 
to v a color not used by any of its neighbors. The difficulty 
arises if v has five neighbors, and each has a different color in 
the 5-coloring of G’. Suppose that the neighbors of v, when 
considered in clockwise order around v, are a, b,c, m, and p. 
(This order is determined by the clockwise order of the curves 
representing the edges incident to v.) Suppose that the colors 
of the neighbors are azure, blue, chartreuse, magenta, and pur- 
ple, respectively. Consider the azure-chartreuse subgraph (i.e., 
the vertices in G colored azure or chartreuse and all the edges 
between them). If a and c are not in the same component of 
this graph, then in the component containing a we can inter- 
change these two colors (make the azure vertices chartreuse 
and vice versa), and G’ will still be properly colored. That 
makes a chartreuse, so we can now color v azure, and G has 
been properly colored. If a and c are in the same component, 
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then there is a path of vertices alternately colored azure and 
chartreuse joining a and c. This path together with edges av 
and ve divides the plane into two regions, with b in one of them 
and m in the other. If we now interchange blue and magenta 
on all the vertices in the same region as b, we will still have a 
proper coloring of G’, but now blue is available for v. In this 
case, too, we have found a proper coloring of G. This com- 
pletes the inductive step, and the theorem is proved. 43. We 
follow the hint. Because the measures of the interior angles of 
a pentagon total 540°, there cannot be as many as three inte- 
rior angles of measure more than 180° (reflex angles). If there 
are no reflex angles, then the pentagon is convex, and a guard 
placed at any vertex can see all points. If there is one reflex 
angle, then the pentagon must look essentially like figure (a) 
below, and a guard at vertex v can see all points. If there are 
two reflex angles, then they can be adjacent or nonadjacent 
(figures (b) and (c)); in either case, a guard at vertex v can see 
all points. [In figure (c), choose the reflex vertex closer to the 
bottom side.] Thus for all pentagons, one guard suffices, so 
(5) = 1, 


Vv 
(a) (b) 


(c) 

45. The figure suggested in the hint (generalized to have k 
prongs for any k > 1) has 3k vertices. The sets of locations 
from which the tips of different prongs are visible are dis- 
joint. Therefore, a separate guard is needed for each of the 
k prongs, so at least k guards are needed. This shows that 
g(3k) > k = [3k/3].Ifn = 3k +i, where 0 <i < 2, then 
a(n) = 93k) > k= [n/3). 


Supplementary Exercises 


1. 2500 3.Yes 5.Yes 7. > ‘"1n; vertices, rej Min 
edges 9%.a)If x e€ MN(A U B), then x is adjacent to 
some vertex v € A U B. WOLOG suppose v € A; then 
x € N(A) and therefore also in N(A) U N(B). Conversely, 
if x € N(A) U N(B), then WOLOG suppose x € N(A). 
Thus x is adjacent to some vertex v € A C AU B, so 
x € N(AUB). b)Ifx € N(AN B), then x is adjacent to 
some vertex v € ANB. Since both v € A andv ¢€ B, it follows 
that x € N(A) and x € N(B), whence x € N(A)N N(B). 
For the counterexample, let G = ({u, v, w}, {{u, v}, {v, w}}), 
A = {u}, and B = {w}. 11. (c, a, p, x, n,m) and many 
others 13. (c,d, a,b) and many others 15. 6 times the 
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number of triangles divided by the number of paths of length 2 
17. a) The probability that two actors each of whom has ap- 
peared in a film with a randomly chosen actor have ap- 
peared in a film together b) The probability that two of a 
randomly chosen person’s Facebook friends are themselves 
Facebook friends c) The probability that two of a randomly 
chosen person’s coauthors are themselves coauthors d) The 
probability that two proteins that each interact with a ran- 
domly chosen protein interact with each other e) The prob- 
ability that two routers each of which has a communica- 
tions link to a randomly chosen router are themselves linked 
19. Complete subgraphs containing the following sets of ver- 
tices: {b, c, e, f}, {a, b, g}, {a, d, g}, {d, e, g}, {b, e, g} 
21. Complete subgraphs containing the following sets of ver- 
tices: {b, c,d, j, k}, {a, b, j, k}, fe, f, g, i}, {a, b, i}, {a, i, 7}, 
{b, d, e}, {b, e, i}, {b, i, J}, {g,h, i}, th, i, j} 23. {c, d}isa 
minimum dominating set. 

25. a) 


Ne, 


27.a)1 b)2 c)3  29.a)A path from u to v in a graph 
G induces a path from f(u) to f(v) in an isomorphic 
graph H. bb) Suppose f is an isomorphism from G to 
HA. If vo, vj, .--, Yn, Vo is a Hamilton circuit in G, then 
fio), £1), ---, fn), fo) must be a Hamilton circuit 
in H because it is still a circuit and f(v;) 4 f(v;) for 
0 <i < j <n. ce) Suppose f is an isomorphism from 
G to H. If vo, v1, ..., Yn, Vo iS an Euler circuit in G, then 
fo), £1), ---, fn), f(vo) must be an Euler circuit in 
H because it is a circuit that contains each edge exactly once. 
d) Two isomorphic graphs must have the same crossing num- 
ber because they can be drawn exactly the same way in the 
plane. e) Suppose f is an isomorphism from G to H. Then 
v is isolated in G if and only if f(v) is isolated in H. Hence, 
the graphs must have the same number of isolated vertices. 


f) Suppose f is an isomorphism from G to H. If G is bipar- 
tite, then the vertex set of G can be partitioned into V; and V2 
with no edge connecting vertices within V; or vertices within 
V2. Then the vertex set of H can be partitioned into f(V) 
and f (V2) with no edge connecting vertices within f (V1) or 
vertices within f(V2). 31.3  33.a)Yes b)No 35. No 
37.Yes 39. If e is a cut edge with endpoints u and v, then 
if we direct e from u to v, there will be no path in the di- 
rected graph from v to u, or else e would not have been a 
cut edge. Similar reasoning works if we direct e from v to uw. 
4i.n—1 43. Let the vertices represent the chickens. We 
include the edge (u, v) in the graph if and only if chicken u 
dominates chicken v. 45. By the handshaking theorem, the 
average vertex degree is 2m /n, which equals the minimum de- 
gree; it follows that all the vertex degrees areequal. 47. K33 
and the skeleton of a triangular prism 49.a)A Hamilton 
circuit in the graph exactly corresponds to a seating of the 
knights at the Round Table such that adjacent knights are 
friends. b) The degree of each vertex in this graph is at least 
2n — 1—(n—1) =n > (2n/2), so by Dirac’s theorem, this 
graph has a Hamilton circuit. c)a,b,d, f,g,z Sl.a)4 
b)2 c)3 d)4 e)4 f)2 53.a) Suppose that G = (V, E). 
Let a, b € V. We must show that the distance between a and 
b in G is at most 2. If {a, b} ¢ E this distance is 1, so assume 
{a, b} € E. Because the diameter of G is greater than 3, there 
are vertices u and v such that the distance in G between u 
and v is greater than 3. Either uw or v, or both, is not in the set 
{a, b}. Assume that u is different from both a and b. Either 
{a, u} or {b, u} belongs to E; otherwise a, u, b would be a 
path in G of length 2. So, without loss of generality, assume 
{a, u} € E. Thus v cannot be a or b, and by the same rea- 
soning either {a, v} € E or {b, v} € E. In either case, this 
gives a path of length less than or equal to 3 from u to v in 
G, a contradiction. b) Suppose G = (V, E). Leta, b € V. 
We must show that the distance between a and b in G does 
not exceed 3. If {a, b} ¢ E, the result follows, so assume that 
{a, b} € E. Because the diameter of G is greater than or equal 
to 3, there exist vertices u and v such that the distance in G 
between u and v is greater than or equal to 3. Either wu or v, or 
both, is not in the set {a, b}. Assume u is different from both 
a and b. Either {a, u} € E or {b, u} € E; otherwise a, u, b 
is a path of length 2 in G. So, without loss of generality, as- 
sume {a,u} € E. Thus v is different from a and from b. If 
{a, v} € E, then uw, a, visapath of length 2in G,so {a, v} ¢ E 
and thus {b, v} € E (or else there would be a path a, v, b of 
length 2 in G). Hence, {u, b} ¢ E; otherwise u, b, v is a path 
of length 2 in G. Thus, a, v, u, b is a path of length 3 in G, 
as desired. 55.a,b,e,z S7.a,c,d, f,g,z 59.If Gis 
planar, then because e < 3v — 6, G has at most 27 edges. 
(If G is not connected it has even fewer edges.) Similarly, G 
has at most 27 edges. But the union of G and G is K1, which 
has 55 edges, and 55 > 27+ 27. 61. Suppose that G is 
colored with k colors and has independence number 7. Be- 
cause each color class must be an independent set, each color 
class has no more than i elements. Thus there are at most ki 


vertices. 63.a) C(n,m)p”(1 — p)”~™ b)np c) To gen- 
erate a labeled graph G, as we apply the process to pairs of 
vertices, the random number x chosen must be less than or 
equal to 1/2 when G has an edge between that pair of vertices 
and greater than 1/2 when G has no edge there. Hence, the 
probability of making the correct choice is 1/2 for each edge 
and 1/2©-2) overall. Hence, all labeled graphs are equally 
likely. 65. Suppose P is monotone increasing. If the prop- 
erty of not having P were not retained whenever edges are 
removed from a simple graph, there would be a simple graph 
G not having P and another simple graph G’ with the same 
vertices but with some of the edges of G missing that has P. 
But P is monotone increasing, so because G’ has P, so does 
G obtained by adding edges to G’. This is a contradiction. The 
proof of the converse is similar. 


CHAPTER 11 
Section 11.1 


1.(a),(c), (ce) 3.a)a b)a,b,c,d, f,h,j.q.t Oe, g,i,k, 
l,m,n,o,p,r,s,u d)g,re)cf)p g) f,b,ahje, f,l,m,n 
5.No 7. Level 0: a; level 1: b, c, d; level 2: e through k (in 
alphabetical order); level 3: / through r; level 4: s, t; level 5: 
u 9. a) The entire tree b)c, g, h, o, p and the four edges 
cg, ch,ho, hp c)ealone ll.a)l b)2 13.a)3 b)9 
15. a) The “only if” part is Theorem 2 and the definition of a 
tree. Suppose G is a connected simple graph with n vertices 
and n — | edges. If G is not a tree, it contains, by Exercise 
14, an edge whose removal produces a graph G’, which is still 
connected. If G’ is not a tree, remove an edge to produce a 
connected graph G”’. Repeat this procedure until the result is 
a tree. This requires at most n — | steps because there are only 
n — 1 edges. By Theorem 2, the resulting graph has n — 1 
edges because it has n vertices. It follows that no edges were 
deleted, so G was already a tree. b) Suppose that G is a tree. 
By part (a), G has n— 1 edges, and by definition, G has no sim- 
ple circuits. Conversely, suppose that G has no simple circuits 
and has n — | edges. Let c equal the number of components 
of G, each of which is necessarily a tree, say with n; vertices, 
where )“*_, nj; =n. By part (a), the total number of edges in 
Gis )_, (nj — 1) =n-—c. Since we are given that this equals 
n — 1, it follows that c = 1, i.e., G is connected and there- 
fore satisfies the definition of a tree. 17.9999 19.2000 
21.999 23. 1,000,000 dollars 25. No such tree exists by 
Theorem 4 because it is impossible for m = 2 or m = 84. 
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27. Complete binary tree of height 4: 


Complete 3-ary tree of height 3: 


29. a) By Theorem 3 it follows thatn = mi + 1. Because 
i+l=n,wehavel =n—i,sol = (mi+1)—i = (m—1)i4+1. 
b) We have n = mi + 1 andi+/ =n. Hence,i = n —1. 
It follows thatn = m(n — 1) + 1. Solving for n gives 
n = (ml 1)/(m 1). From i = n — 1 we obtain 
i=([@ml—1)/™m—1))-l1=@d-D/(m-1). 31.n-t 
33.a) 1 b)3 ¢)5 35. a) The parent directory b) A subdi- 
rectory or contained file c) A subdirectory or contained file in 
the same parent directory d) All directories in the path name 
e) All subdirectories and files continued in the directory or 
a subdirectory of this directory, and so on f) The length of 
the path to this directory or file g) The depth of the system, 
i.e., the length of the longest path 37.Letn = 2", where 
k is a positive integer. If k = 1, there is nothing to prove 
because we can add two numbers with n — | = | processor 
in log2 = 1 step. Assume we can add n = 2* numbers in 
logn steps using a tree-connected network of n — 1 proces- 
sors. Let x}, x2, ..., X2n be 2n = 2*+! numbers that we 
wish to add. The tree-connected network of 2n — 1 processors 
consists of the tree-connected network of n — 1 processors 
together with two new processors as children of each leaf. In 
one step we can use the leaves of the larger network to find 
xX + .x2,%3 + X4,..., X2n—-1 + X2n, giving us n numbers, 
which, by the inductive hypothesis, we can add in logn steps 
using the rest of the network. Because we have used logn + 1 
steps and log(2n) = log2+ logn = 1+ logn, this completes 
the proof. 39.conly 41.candh 43. Suppose a tree T 
has at least two centers. Let u and v be distinct centers, both 
with eccentricity e, with u and v not adjacent. Because T is 
connected, there is a simple path P from u to v. Let c be any 
other vertex on this path. Because the eccentricity of c is at 
least e, there is a vertex w such that the unique simple path 
from c to w has length at least e. Clearly, this path cannot 
contain both u and v or else there would be a simple circuit. 
In fact, this path from c to w leaves P and does not return 
to P once it, possibly, follows part of P toward either u or v. 
Without loss of generality, assume this path does not follow 
P toward u. Then the path from u to c to w is simple and of 
length more than e, a contradiction. Hence, u and v are adja- 
cent. Now because any two centers are adjacent, if there were 
more than two centers, T would contain K3, a simple circuit, 
as a subgraph, which is a contradiction. 
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45. T, Ty 


7 


47. The statement is that every tree with n vertices has a path 
of length n — 1, and it was shown only that there exists a tree 
with n vertices having a path of length n — 1. 


Section 11.2 


1. banana 


papaya 
3. a)3_ b)1 c)4 d)5 


5. the 


dog 


lazy 


7. At least [log, 4] = 2 weighings are needed, because there 
are only four outcomes (because it is not required to determine 
whether the coin is lighter or heavier). In fact, two weighings 
suffice. Begin by weighing coin | against coin 2. If they bal- 
ance, weigh coin | against coin 3. If coin | and coin 3 are the 
same weight, coin 4 is the counterfeit coin, and if they are not 
the same weight, then coin 3 is the counterfeit coin. If coin 1 
and coin 2 are not the same weight, again weigh coin | against 
coin 3. If they balance, coin 2 is the counterfeit coin; if they 
do not balance, coin | is the counterfeit coin. 9. At least 


[log3 13] = 3 weighings are needed. In fact, three weighings 
suffice. Start by putting coins 1, 2, and 3 on the left-hand side 
of the balance and coins 4, 5, and 6 on the right-hand side. 
If equal, apply Example 3 to coins 1, 2, 7, 8, 9, 10, 11, and 
12. If unequal, apply Example 3 to 1, 2, 3, 4, 5, 6, 7, and 8. 
11. The least number is five. Call the elements a, b, c, and d. 
First compare a and b; then compare c and d. Without loss of 
generality, assume thata < bandc < d. Next compare a 
and c. Whichever is smaller is the smallest element of the set. 
Again without loss of generality, suppose a < c. Finally, com- 
pare b with both c and d to completely determine the ordering. 
13. The first two steps are shown in the text. After 22 has been 
identified as the second largest element, we replace the leaf 
22 by —oo in the tree and recalculate the winner in the path 
from the leaf where 22 used to be up to the root. Next, we see 
that 17 is the third largest element, so we repeat the process: 
replace the leaf 17 by —oo and recalculate. Next, we see that 
14 is the fourth largest element, so we repeat the process: re- 
place the leaf 14 by —oo and recalculate. Next, we see that 11 
is the fifth largest element, so we repeat the process: replace 
the leaf 11 by —oo and recalculate. The process continues in 
this manner. We determine that 9 is the sixth largest element, 
8 is the seventh largest element, and 3 is the eighth largest 
element. The trees produced in all steps, except the second to 
last, are shown here. 


15. The value of a vertex is the list element currently there, 
and the label is the name (i.e., location) of the leaf responsible 
for that value. 
procedure tournament sort(da,, ..., Qn) 
k := [flogn] 
build a binary tree of height k 
for i := 1ton 
set the value of the ith leaf to be a; and its label to 
be itself 
for i := n+ 1 to 2‘ 
set the value of the ith leaf to be —oo and its label to 
be itself 
for i := k — 1 downto 0 
for each vertex v at level i 
set the value of v to the larger of the values of its 
children and its label to be the label of the child 
with the larger value 
for i := 1 ton 
cj 1= value at the root 
let v be the label of the root 
set the value of v to be —oo 
while the label at the root is still v 
Vv != parent(v) 
set the value of v to the larger of the values of its 
children and its label to be the label of the child 
with the larger value 
{c1,..., Cy is the list in nonincreasing order} 
17.k — 1, wheren = 2* 19.a)Yes b)No c) Yes d) Yes 
21. a: 000, e: 001, 7: 01, k: 1100, 0: 1101, p: 11110, uw: 11111 
23. a: 11; b: 101; c: 100; d: 01; e: 00; 2.25 bits (Note: This 
coding depends on how ties are broken, but the average num- 
ber of bits is always the same.) 25. There are four possible 
answers in all, the one shown here and three more obtained 
from this one by swapping t and v and/or swapping u and w. 
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27. A:0001; B:101001; C:11001; D:00000; —_E:100; 
F:001100; G:001101; H:0101; 1:0100; J:110100101; 
K:1101000; L:00001; M:10101; N:0110; 0:0010; P:101000; 
Q:1101001000; R:1011; S:0111; T:111; U:00111; V:110101; 
W:11000; X:11010011; Y:11011; Z:1101001001 29. A:2; 
E:1;N:010; R:011;T:02;Z:00 31. 33. Because the tree 
is rather large, we have indicated in some places to “see text.” 
Refer to Figure 9; the subtree rooted at these square or circle 
vertices is exactly the same as the corresponding subtree in 
Figure 9. First player wins. 


+1 


max 


see 
text 
+1 


35. a) $1 b) $3 c)—$3 37. See the figures shown next. 
a)0 b)O c)1 d) This position cannot have occurred in a 
game; this picture is impossible. 


0 0 

o|x|x xlo|x 

2» xlolo ) olx|x 

X O 
or a 
o|x|x o|x|x  xlolx xjo|x 
xlolo xlolo ‘olxlx Olx[x 
o| [x olx olo of fo 
‘| R al lo 
o|x|x o|x|x  xlolx xlolx 
xlolo xlolo Olxlx O[x!x 
olx|x xlolx xlolo olxlo 
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+1 
X O 
o|0 
x Xx 
i 
X|X|O x O xX O 
O/O O/O;|X O/;/O 
x xX Xx X X|X|X 
re a ie Se cilia 
X|X}O X|XjO X/jJO;/O xX O 
O|;O;/O OjO O;O|X OjO;|X 
Xx xX X{O|X xX X X/O|X 
O wins o| | ‘| 
X|X/O Xj/O;JO X{xXjO 
OJO;X O;O;X O;O;X 
X|O;X XIX|X XIlO}X 
draw X wins draw 


39. Proof by strong induction: Basis step: When there are 
n = 2 stones in each pile, if first player takes two stones 
from a pile, then second player takes one stone from the re- 
maining pile and wins. If first player takes one stone from 
a pile, then second player takes two stones from the other 
pile and wins. Inductive step: Assume inductive hypothesis 
that second player can always win if the game starts with two 
piles of 7 stones for all 2 < j < k, where k > 2, and 
consider a game with two piles containing k + 1 stones each. 
If first player takes all the stones from one of the piles, then 
second player takes all but one stone from the remaining pile 
and wins. If first player takes all but one stone from one of the 
piles, then second player takes all the stones from the other pile 
and wins. Otherwise first player leaves j stones in one pile, 
where 2 < j < k, andk + 1 stones in the other pile. Second 
player takes the same number of stones from the larger pile, 
also leaving 7 stones there. At this point the game consists of 
two piles of j stones each. By the inductive hypothesis, the 
second player in that game, who is also the second player in 
our actual game, can win, and the proof by strong induction 
is complete. 41.7; 49 43. Value of tree is 1. Note: The 
second and third trees are the subtrees of the two children of 
the root in the first tree whose subtrees are not shown because 
of space limitations. They should be thought of as spliced into 
the first picture. 


+1 =l 
xX 
5 aes Pe Ale 
Xx Xx X]O xX| |o X xX Xx 
xX (e) {e) 
(e) 
Cas 6=S=1 5=-5=0 4-S=-1 6-5=1 5-5=0 
ie) ie) 
xX X 
5-4=1 6-4=2 
2 
xX 
X x x|O X X 
[e) (e) 
lo) le) 
4-6=-2 6-6=0 5-6=-1 6-6=0 5-6=-1 
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2 3 


1.1 12 
0<1<1.1<1.2<2<3 


a 0 


1.2.1.1 


1.2.3.2.1 1.2.3.2.2 
5.No 7.a,b,d,e, f,g,c 9%.a,b,e,k, l,m, f, g,n,7, 


s,c,d,h,o,i,j,p,q  \1.d,b,i,e,m, j,n,o,a, f,c, g,k, 
h,p,l 13.d, f,g,e,b,c,a 15.k,l,m,e, f,r,s,n, g,b, 
c,o,h,i, p,q, j,d,a 


17. a) + 


es 


b) ++x«xy/xy,+x/+xxyxy ce) xxyetxy/+,xxyaxty/+ 
d) (x + & * y)) + /y)), (& + C& * y) + x)/Y)) 


b)—NABUA-BA ce) ABNABA-—U- 


d) (AN B)—(AU(B—A))) 21.14 23.a)1 b)1 o4 
d) 2205 
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27. Use mathematical induction. The result is trivial for a list 
with one element. Assume the result is true for a list with 
n elements. For the inductive step, start at the end. Find the 
sequence of vertices at the end of the list starting with the 
last leaf, ending with the root, each vertex being the last child 
of the one following it. Remove this leaf and apply the in- 
ductive hypothesis. 29.c,d,b, f, g, h, e, a in each case 
31. Proof by mathematical induction. Let S(X) and O(X) rep- 
resent the number of symbols and number of operators in the 
well-formed formula X, respectively. The statement is true for 
well-formed formulae of length 1, because they have | sym- 
bol and 0 operators. Assume the statement is true for all well- 
formed formulae of length less than n. A well-formed formula 
of length n must be of the form «XY, where « is an operator 
and X and Y are well-formed formulae of length less than n. 
Then by the inductive hypothesis S(«XY) = S(X)+ S(Y) = 
[O(X) + 1] + [O(Y) + 1] = O(X) + O(Y) + 2. Because 
O(*XY) = 1+ O(X) + O(Y), it follows that S(«#XY) = 
OXY) + 1. 33.xy4+zx0+x0, xyz ++ yxt4, 
xXyxyooxyoozo+, xzZX, ZZ+0, yyyyooo, zx+yz+o, for in- 
stance 


Section 11.4 
l.m-—n+1 
3,4 b c 
d 
g ft e 
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9, 4 b f g a b f g a b f g a b f g 
c d e h c d e h c d e h ie d e h 
a b f g a b f g a b f g a b f g 
| | | | @ e 
e @ e e @ e 
c d e h é d e h c d e h c d e h 
a b f g a b f g a b f g a b f g 

@ @ e e @ @ 
e d e h c d e h c d e h e d e h 
a b f g a b f g a b f g a b f g 
c d e h ral d e h c d e h c d e h 


iS; 
a 


17. a)A path of length 6 b)A path of length 5 c)A path 
of length 6 d) Depends on order chosen to visit the vertices; 
may bea path of length 7 19. With breadth-first search, the 
initial vertex is the middle vertex, and the n spokes are added 
to the tree as this vertex is processed. Thus, the resulting tree 
is K,,,. With depth-first search, we start at the vertex in the 
middle of the wheel and visit a neighbor—one of the vertices 
on the rim. From there we move to an adjacent vertex on the 
rim, and so on all the way around until we have reached every 
vertex. Thus, the resulting spanning tree is a path of length n. 
21. With breadth-first search, we fan out from a vertex of de- 
gree m to all the vertices of degree n as the first step. Next, 
a vertex of degree n is processed, and the edges from it to all 


the remaining vertices of degree m are added. The result is 
a K,,,-1 and a Kj, ,—1 with their centers joined by an edge. 
With depth-first search, we travel back and forth from one 
partite set to the other until we can go no further. Ifm = n 
orm =n — I, then we get a path of length m + n — 1. Oth- 
erwise, the path ends while some vertices in the larger partite 
set have not been visited, so we back up one link in the path to 
a vertex v and then successively visit the remaining vertices in 
that set from v. The result is a path with extra pendant edges 
coming out of one end of the path. 23. A possible set of 
flights to discontinue are: Boston—New York, Detroit—Boston, 
Boston—Washington, New York—Washington, New York-— 
Chicago, Atlanta—Washington, Atlanta—Dallas, Atlanta—Los 
Angeles, Atlanta—St. Louis, St. Louis—Dallas, St. Louis— 
Detroit, St. Louis—Denver, Dallas—San Diego, Dallas—Los An- 
geles, Dallas—-San Francisco, San Diego—Los Angeles, Los 
Angeles—San Francisco, San Francisco—Seattle. 25. Proof 
by induction on the length of the path: If the path has length 0, 
then the result is trivial. If the length is 1, then w is adjacent to 
v, So u is at level 1 in the breadth-first spanning tree. Assume 
that the result is true for paths of length /. If the length of a 
path is 7 + 1, let w’ be the next-to-last vertex in a shortest 
path from v to u. By the inductive hypothesis, u’ is at level 
/ in the breadth-first spanning tree. If uw were at a level not 
exceeding /, then clearly the length of the shortest path from 
v to u would also not exceed /. So u has not been added to 
the breadth-first spanning tree yet after the vertices of level / 
have been added. Because u is adjacent to uv’, it will be added 
at level / + 1 (although the edge connecting u’ and u is not 
necessarily added). 27. a) No solution 


fas: 


a 


29. Start at a vertex and proceed along a path without repeat- 
ing vertices as long as possible, allowing the return to the start 
after all vertices have been visited. When it is impossible to 
continue along a path, backtrack and try another extension of 
the current path. 31. Take the union of the spanning trees 
of the connected components of G. They are disjoint, so the 
resultisaforest. 33.m—n-+c 35. Assume that we wish 
to find the length of a shortest path from v; to every other 
vertex of G using Algorithm 1. In line 2 of that algorithm, add 
L(1) := 0, and add the following as a third step in the then 
clause at the end: L(w) :=1+L(v). 37. Add an instruction 
to the BFS algorithm to mark each vertex as it is encoun- 
tered. When BFS terminates we have found (all the vertices 
of) one component of the graph. Repeat, starting at an un- 
marked vertex, and continue in this way until all vertices 
have been marked. 39.Trees 41. Use depth-first search 
oneachcomponent. 43. If an edge wv is not followed while 
we are processing vertex u during the depth-first search pro- 
cess, then it must be the case that the vertex v had already been 
visited. There are two cases. If vertex v was visited after we 
started processing u, then, because we are not finished pro- 
cessing u yet, v must appear in the subtree rooted at wu (and 
hence, must be a descendant of uw). On the other hand, if the 
processing of v had already begun before we started process- 
ing u, then why wasn’t this edge followed at that time? It must 
be that we had not finished processing v, in other words, that 
we are still forming the subtree rooted at v, so u is a descendant 
of v,andhence, visanancestorofu. 45. Certainly these two 
procedures produce the identical spanning trees if the graph 
we are working with is a tree itself, because in this case there 
is only one spanning tree (the whole graph). This is the only 
case in which that happens, however. If the original graph has 
any other edges, then by Exercise 43 they must be back edges 
and hence, join a vertex to an ancestor or descendant, whereas 
by Exercise 34, they must connect vertices at the same level 
or at levels that differ by 1. Clearly these two possibilities 
are mutually exclusive. Therefore there can be no edges other 
than tree edges if the two spanning trees are to be the same. 
47. Because the edges not in the spanning tree are not followed 
in the process, we can ignore them. Thus we can assume that 
the graph was a rooted tree to begin with. The basis step is 
trivial (there is only one vertex), so we assume the inductive 
hypothesis that breadth-first search applied to trees with n ver- 
tices have their vertices visited in order of their level in the 
tree and consider a tree T with n + 1 vertices. The last vertex 
to be visited during breadth-first search of this tree, say v, is 
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the one that was added last to the list of vertices waiting to 
be processed. It was added when its parent, say u, was being 
processed. We must show that v is at the lowest (bottom-most, 
i.e., numerically greatest) level of the tree. Suppose not; say 
vertex x, whose parent is vertex w, is at a lower level. Then w 
is at a lower level than uw. Clearly v must be a leaf, because any 
child of v could not have been seen before v is seen. Consider 
the tree T’ obtained from T by deleting v. By the inductive 
hypothesis, the vertices in T’ must be processed in order of 
their level in T’ (which is the same as their level in T, and the 
absence of v in T’ has no effect on the rest of the algorithm). 
Therefore u must have been processed before w, and therefore 
v would have joined the waiting list before x did, a contradic- 
tion. Therefore v is at the bottom-most level of the tree, and 
the proof is complete. 49. We modify the pseudocode given 
in Algorithm 2 by initializing m to be 0 at the beginning of 
the algorithm, and adding the statements “m := m + 1” and 
“assign m to vertex v” after the statement that removes vertex 
vfrom L. 51. Ifa directed edge wv is not followed while we 
are processing its tail uw during the depth-first search process, 
then it must be the case that its head v had already been visited. 
There are three cases. If vertex v was visited after we started 
processing u, then, because we are not finished processing u 
yet, v must appear in the subtree rooted at u (and hence, must 
be a descendant of uw), so we have a forward edge. Otherwise, 
the processing of v must have already begun before we started 
processing u. If it had not yet finished (i.e., we are still forming 
the subtree rooted at v), then u is a descendant of v, and hence, 
v is an ancestor of u (we have a back edge). Finally, if the pro- 
cessing of v had already finished, then by definition we have 
across edge. 53. Let T be the spanning tree constructed in 
Figure 3 and 7), 77, 73, and T, the spanning trees in Figure 4. 
Denote by d(T’, T”) the distance between T’ and T”. Then 
d(T, 7) = 6, d(T, To) = 4, d(T, T3) = 4, d(T, Ts) = 2, 
d(T, Tr) = 4, d(T, T3) = 4, d(N, Ts) = 6, d(Th, T3) = 4, 
d(T2, Ty) = 2, and d(T3, T4) = 4. 55. Suppose e; = {u, v} 
is as specified. Then 7> U {e;} contains a simple circuit C 
containing e;. The graph 7; — {e;} has two connected compo- 
nents; the endpoints of e; are in different components. Travel 
C from u in the direction opposite to e; until you come to the 
first vertex in the same component as v. The edge just crossed 
is e2. Clearly, T> U {e1} — {e2} is a tree, because e2 was on C. 
Also 7; — {e1} U {e2} is a tree, because e2 reunited the two 
components. 


57. Exercise 18: Exercise 19: Exercise 20: 
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Exercise 21: Exercise 22: Exercise 23: 


a b c a b c c fa i I 


e fe 4d 4 dg J 
59. First construct an Euler circuit in the directed graph. Then 
delete from this circuit every edge that goes to a vertex pre- 
viously visited. 61. According to Exercise 60, a directed 
graph contains a circuit if and only if there are any back edges. 
We can detect back edges as follows. Add a marker on each 
vertex v to indicate what its status is: not yet seen (the initial 
situation), seen (i.e., put into 7) but not yet finished (i.e., 
visit(v) has not yet terminated), or finished (i.e., visit(v) has 
terminated). A few extra lines in Algorithm 1 will accomplish 
this bookkeeping. Then to determine whether a directed graph 
has a circuit, we just have to check when looking at edge uv 
whether the status of v is “seen.” If that ever happens, then we 
know there is a circuit; if not, then there is no circuit. 


Section 11.5 


1. Deep Springs—Oasis, Oasis—Dyer, Oasis—Silver Peak, Sil- 
ver Peak—Goldfield, Lida—Gold Point, Gold Point—Beatty, 
Lida—Goldfield, Goldfield-Tonopah, Tonopah—Manhattan, 
Tonopah—Warm Springs _ 3. {e, f}, {c, f}, {e, A}, {A, i}, 
{b, c}, {b, d}, {a, d}, {g, h} 


5. ; Chicago New York 
San Francisco $120 


Atlanta 


7. {e, f}, {a, d}, {h, i}, {b, d}, fe, f}, {eh}, {b,c}, {g, A} 


11. Instead of choosing minimum-weight edges at each stage, 
choose maximum-weight edges at each stage with the same 
properties. 


13. a b 
3 
dae} 
€ 
c d 


17. First find a minimum spanning tree T of the graph G with 
n edges. Then fori = 1 ton — 1, delete only the ith edge of 
T from G and find a minimum spanning tree of the remaining 


graph. Pick the one of these n — | trees with the shortest length. 
19. If all edges have different weights, then a contradiction is 
obtained in the proof that Prim’s algorithm works when an 
edge ex, is added to T and an edge e is deleted, instead of 
possibly producing another spanning tree. 


21. @2 53 ¢ 1 d 


@ 
3 1 2 
e fi 3 h 
4 2 4 
1 
i J k 


23. Same as Kruskal’s algorithm, except start with T := this 
set of edges and iterate from i = 1 toi =n — 1 — 5, wheres 
is the number of edges you start with. 


25. a) Chicago New York 
San Francisco _ $1200 


27. By Exercise 24, at each stage of Sollin’s algorithm a forest 
results. Hence, after n — 1 edges are chosen, a tree results. It 
remains to show that this tree is a minimum spanning tree. 
Let T be a minimum spanning tree with as many edges in 
common with Sollin’s tree S as possible. If T # S, then there 
is an edge e € S — T added at some stage in the algorithm, 
where prior to that stage all edges in S are also in T. T U {e} 
contains a unique simple circuit. Find an edge e’ € S — T 
and an edge e” € T — S on this circuit and “adjacent” when 
viewing the trees of this stage as “‘supervertices.” Then by the 
algorithm, w(e’) < w(e”). So replace T by T — {e”} U {e’} 
to produce a minimum spanning tree closer to S than T was. 
29. Each of the r trees is joined to at least one other tree by 
a new edge. Hence, there are at most r/2 trees in the result 
(each new tree contains two or more old trees). To accomplish 
this, we need to add r — (r/2) = r/2 edges. Because the 
number of edges added is integral, itis at least [r/2]. 31. If 
k > logan, then n/2* < 1, so [n/2*] = 1, so by Exer- 
cise 30 the algorithm is finished after at most log n iterations. 
33. Suppose that a minimum spanning tree T contains edge 
e = uv that is the maximum weight edge in simple circuit C. 
Delete e from 7. This creates a forest with two components, 
one containing u and the other containing v. Follow the edges 
of the path C — e, starting at u. At some point this path must 
jump from the component of T — e containing u to the com- 
ponent of T — e containing v, say using edge f. This edge 
cannot be in T, because e can be the only edge of T joining 
the two components (otherwise there would be a simple cir- 
cuit in T). Because e is the edge of greatest weight in C, the 


weight of f is smaller. The tree formed by replacing e by f 
in T therefore has smaller weight, a contradiction. 35. The 
reverse-delete algorithm must terminate and produce a span- 
ning tree, because the algorithm never disconnects the graph 
and upon termination there can be no more simple circuits. 
The edge deleted at each stage of the algorithm must have 
been the edge of maximum weight in whatever circuits it was 
a part of. Therefore by Exercise 33 it cannot be in any mini- 
mum spanning tree. Since only edges that could not have been 
in any minimum spanning tree have been deleted, the result 
must be a minimum spanning tree. 


Supplementary Exercises 


1. Suppose T is a tree. Then clearly T has no simple circuits. 
If we add an edge e connecting two nonadjacent vertices u and 
v, then obviously a simple circuit is formed, because when e 
is added to T the resulting graph has too many edges to be a 
tree. The only simple circuit formed is made up of the edge e 
together with the unique path in T from v to u. Suppose T sat- 
isfies the given conditions. All that is needed is to show that T 
is connected, because there are no simple circuits in the graph. 
Assume that T is not connected. Then let u and v be in separate 
connected components. Adding e = {u, v} does not satisfy 
the conditions. 3. Suppose that a tree T has n vertices of 
degrees d}, dy, ..., dn, respectively. Because 2e = )7"_, dj 
and e = n — 1, we have 2(n — 1) = )~_, d;. Because each 
d; = 1, it follows that 2(n — 1) = n+ >~¥_, (dj — 1), or that 
n—2=)~/_,(d; — 1). Hence, at most n — 2 of the terms of 
this sum can be | or more. Hence, at least two of them are 0. 
It follows that d; = 1 for at least two values of i. 5.2n —2 
7. A tree has no circuits, so it cannot have a subgraph homeo- 
morphic to K3.30r Ks. 9. Color each connected component 
separately. For each of these connected components, first root 
the tree, then color all vertices at even levels red and all ver- 
tices at odd levels blue. 11. Upper bound: k"; lower bound: 
2[k / 2] h-1 

13. 

By By Bo 


PhP att 


15. Because By.) is formed from two copies of Bx, one shifted 
down one level, the height increases by | as k increases by 1. 
Because Bo had height 0, it follows by induction that By has 
height k. 17. Because the root of By+1 is the root of By 
with one additional child (namely the root of the other B;), 
the degree of the root increases by 1 as k increases by 1. Be- 
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cause Bo had a root with degree 0, it follows by induction that 
By has a root with degree k. 


19; © 
So Sy Sy 53 


S4 


21. Use mathematical induction. The result is trivial for k = 0. 
Suppose it is true for k — 1. T,_ is the parent tree for T. By in- 
duction, the child tree for T can be obtained from Jo, ... , Tk—2 
in the manner stated. The final connection of rg_2 to rp_} is 
as stated in the definition of S;-tree. 
23. procedure /evel(T: ordered rooted tree with root r) 
queue := sequence consisting of just the root r 
while queue contains at least one term 
v := first vertex in queue 
list v 
remove v from queue and put children of v onto 
the end of queue 


25. Build the tree by inserting a root for the address 0, and 
then inserting a subtree for each vertex labeled 7, for i a posi- 
tive integer, built up from subtrees for each vertex labeled i. j 
for j a positive integer, andsoon. 27. a) Yes b) No c) Yes 
29. The resulting graph has no edge that is in more than one 
simple circuit of the type described. Hence, it is a cactus. 


31a be a 


37.6 39. a) Ofor00, 11 for01, 100 for 10, 101 for 11 (exact 
coding depends on how ties were broken, but all versions are 
equivalent); 0.645n for string of lengthn b) 0 for 000, 100 for 
001, 101 for 010, 110 for 100, 11100 for 011, 11101 for 101, 
11110 for 110, 11111 for 111 (exact coding depends on how 
ties were broken, but all versions are equivalent); 0.5326n for 
string oflengthn 41. Let G’ be the graph obtained by delet- 
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ing from G the vertex v and all edges incident to v. A minimum 
spanning tree of G can be obtained by taking an edge of min- 
imal weight incident to v together with a minimum spanning 
treeofG’. 43. Suppose that edge ¢ is the edge of least weight 
incident to vertex v, and suppose that T is a spanning tree that 
does not include e. Add e to T, and delete from the simple cir- 
cuit formed thereby the other edge of the circuit that contains 
v. The result will be a spanning tree of strictly smaller weight 
(because the deleted edge has weight greater than the weight of 
e). This is acontradiction, so T mustincludee. 45. Because 
paths in trees are unique, an arborescence T of a directed graph 
G is just a subgraph of G that is a tree rooted at 7, containing 
all the vertices of G, with all the edges directed away from the 
root. Thus the in-degree of each vertex other than r is 1. For 
the converse, itis enough to show that for each v € V there isa 
unique directed path from r to v. Because the in-degree of each 
vertex other than r is 1, we can follow the edges of T back- 
wards from v. This path can never return to a previously visited 
vertex, because that would create a simple circuit. Therefore 
the path must eventually stop, and it can stop only at r, whose 
in-degree is not necessarily 1. Following this path forward 
gives the path from r to v required by the definition of ar- 
borescence. 47. a) Run the breadth-first search algorithm, 
starting from v and respecting the directions of the edges, 
marking each vertex encountered as reachable. b) Running 
breadth-first search on G°°”", again starting at v, respecting 
the directions of the edges, and marking each vertex encoun- 
tered, will identify all the vertices from which v is reachable. 
c) Choose a vertex v; and using parts (a) and (b) find the strong 
component containing v,, namely all vertices w such that w 
is reachable from v; and v, is reachable from w. Then choose 
another vertex v2 not yet in a strong component and find the 
strong component of v2. Repeat until all vertices have been in- 
cluded. The correctness of this algorithm follows from the def- 
inition of strong component and Exercise 17 in Section 10.4. 


CHAPTER 12 


Section 12.1 


l.a)l b)1 CO ADO 3.a0-1)N+0-14+0) = 
14+(04+0=14+0+0) =14+1=1 b(TATV 
(A(FAT)VF)=T 
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oorr CCOCO°O 


1 
1 
0 
0 
1 
1 
0 
0 
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xyY+Xyz 
1 1] 1 0 
1 1 | 0 1 
1;0]1 1 
1;0;]0 1 
Oo; 1]1 1 
0}; 1/0 1 
O0;0];1 1 
0}0)]0 1 
7. a) 110 


000 001 


9. (0, 0) and (1, 1) 11.x 
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23.0-0=0-1=0;1-1=1-0=0 


al 

1 : 
1 1 
1 0 


27. a) True, as a table of values can show b) False; take x = 1, 
y = 1,z= 1, forinstance c) False; take x = 1, y = 1,z=0, 
for instance 29. By De Morgan’s laws, the complement of 
an expression is like the dual except that the complement of 


each variable has been taken. 31.16 33. If we replace 
each 0 by F, 1 by T, Boolean sum by v, Boolean product 
by A, and” by — (and x by p and y by q so that the vari- 
ables look like they represent propositions, and the equals 
sign by the logical equivalence symbol), then xy = X + y 
becomes =(p A g) = —p V —q and x + y = X y becomes 
(pV q)=—7pAn7q. 35. By the domination, distributive, 
and identity laws, x Vx = (xVx)A1= (xVxX)A(XVX) = XV 
(x AX) =xV0=x. Similarly, x Ax=(x Ax) VO=(xXA 
X)V(XAX) =X A (XVX)=xXAL=x. 37. Because 
Ov 1= 1and0OA 1 = O by the identity and commutative 
laws, it follows that 0 = 1. Similarly, because 1 v 0 = 1 and 
1A0= 1, itfollowsthatl =0. 39. First, note that x A0 = 0 
and x V 1 = 1 for all x, as can easily be proved. To prove the 
first identity, it is sufficient to show that (x V y) V (x Ay) =1 
and (x V y) A(x A y) = 0. By the associative, commutative, 
distributive, domination, and identity laws, (x V y)V (Ay) = 
y VV@ AVI =yVIEVAIAGVH) = yV[LA 
aZAVYJ=yVAVY=OVYVx=1Vx=1and(xv 
VYABAY)=VALXAAVYI=VALHAX)VAAYI=VA 
[OV (XA y)] = YAAyY) =X A (vy A Y) = XAO=0. The 
second identity is proved in a similar way. 41. Using the 
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hypotheses, Exercise 35, and the distributive law it follows 
thatx =xVO=xVXVyY)=AVX)VyY=xVY=0. 
Similarly, y = 0. To prove the second statement, note that 
X=XAL=XA(KAY)=(KAX)AY=XAy = 1. Similarly, 
y=1. 43. Use Exercises 39 and 41 in the Supplementary 
Exercises in Chapter 9 and the definition of a complemented, 
distributed lattice to establish the five pairs of laws in the 
definition. 


Section 12.2 


l.a)x yz b)xyz c)xyz d)xyzZ 3.a)xyz+xyz+ 
xyz txyzt+xyzt+ xyz + xyz b)xyz + xyzZ + Xyz 
cC)xyz + xyz + xyz + xyz d)xyz+ xyz 5S.wxyz+ 
wxyz + wxyz + Wxyz + WxyZ+ WX y2z+ WxryZ + 
wxyz TZax+y+z bDxt+yt+z2oxt+yt+z 
Dy +yo2+ +--+ + yn = Oif and only if y; = 0 fori = 1, 
2,...,n. This holds if and only if x; = 0 when y; = x; and 
x; = |wheny; = x;. ll.ax+y+z b(«+y+z) 
&Z@tytDatyt+aewe+ytwyatyt+zZ Ow+ 
ytz2axatyt+zatyt+zat+yt+z) d)(wt+y+z) 
&@+tyt+tDatytgaat+y+ODe+y+yart 


y+zZ) Beaxtytz Dxt+h+@4+2] O@+Y) 
d)[x+(«+y+2)] 


|y [sy | ete | yty | Gd Ory 


1 0 
0 ; 0 i 
1| 0 1 0 
0} 0 1 


I7.ay{[@ | x) | Wy) | 
Zl2 bDL@INIGIDIIM | Le lx 1 @l ally} 


Ie#|x) | Ol wb | 
cx adAi[x| Go| y)] |x| @]|y)] _ 19. Itis impossible to 


represent X¥ using + and - because there is no way to get the 
value 0 if the input is 1. 
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17. 7.a) yZ yz yz Vz 
x 1 
x 
Circuit 
from H—»> Sum=xQ@y - 
x 
; 15d) 
b) yz yz yz yz 
Circuit * 
from t—> Carry = xy 
15 (c) 
x 1 1 
19, “) —— 
c; ———> c) a4 yz yd 4 
X3 ———?> 
co —-Do—) : x 1 1 1 
Cc, ——— 
c = >> 
cy ———> 1 
cy; —Do— “ : 
x, ———bl_ 
Co ae ee, 
C »| 7 , : prime i i- 
res ) 9, Implicants: XYZ, XYZ, x y z, XYZ, xy, XZ, YZ; prime impli 
cants: xy, XZ, yz; essential prime implicants: xy, xZ, yz 
Section 12.4 on \E 52 5 
1. a) y ny b) xy and x y x} 1 1 1 
x 
& i} 
x 1 


11. The 3-cube on the right corresponds to w; the 3-cube given 


3.a y 
) : : by the top surface of the whole figure represents x; the 3-cube 


; 1 given by the back surface of the whole figure represents y; the 
3-cube given by the right surfaces of both the left and the right 

= 3-cube represents z. In each case, the opposite 3-face repre- 

" sents the complemented literal. The 2-cube that represents wz 
is the right face of the 3-cube on the right; the 2-cube that 

b) y y c) y y represents Xy is bottom rear; the 2-cube that represents yZ is 

front left. 

x 1 x 1 1 

E 1 a 1 1 

5.a) yz yz yz Vz 
x 
x 1 
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13. a) 


Wx 


wx 


ye 


b) Wxyz, WXYZ, WXYZ, WXYZ 


15. a) 
X3X 4X5 XZX4Xs5 AXZX 4X5 AZX 4X5 X3X4X5 XZX4X5 AZXy4X5 XZX4X5 
it | 1 
XX 
XX 
XjX> 
XZX 4X5 XZX 4X5 XZX4X5 XZX 4X5 AX3X 4X5 X3X 4X5 X4XyXs5 X4X4X5 
XX 
XX 
Xi%2] 1 1 
XjX2} 1 1 
c) 
X4X 4X5 XZX4X5 AZX4Xs5 AZX4Xs5 AX 4X5 AZX 4X5 XZX4X5 AZX4X5 
wig) 1 1 1 
XX 
XX 
eee f 1 1 1 


d) 
X3X 4X5 XX 4X5 AZXyX5 AZXyX5 X3X4X5 X4X 4X5 XZX 4X5 XZX 4X5 
See 1 1 
X 1X2 1 1 
EX 1 I 
X 1X2 1 1 


e) X4X 4X5 XZX4X5 XZX4Xs5 AZX4Xs5 AX 4X5 AZX 4X5 AZX4X5 XZX4X5 


Mytz) 1 1 1 
Hy02) 1 1 1 1 
Mike) 1 1 1 
Kiko) 1 1 1 1 


f) X3X 4X5 AX3X4X5 XZX4X5 AXZX 4X5 A3X4X5 X4X 4X5 XZX 4X5 XZX 4X5 


xX 1 1 1 1 
XX 1 1 1 1 
XiX9 1 1 1 1 
X1Xo 1 1 1 1 


17. a)64 b)6 19. Rows | and 4 are considered adjacent. 
The pairs of columns considered adjacent are: columns | and 
4, 1 and 12, 1 and 16, 2 and 11, 2 and 15, 3 and 6, 3 and 10, 4 
and 9, 5 and 8, 5 and 16, 6 and 15, 7 and 10, 7 and 14, 8 and 
13, 9 and 12, 11 and 14, 13 and 16. 


21. Smith 
Jones 
Adams 


—?P 
——P 
—> 
Smith ———» 
Jones >| 
—=P 
—> 
—>) 
—> 


Burton 


Marcus 
Adams 
Burton 


23. a)xz bby Oxz+xz+ yz Dazt+xy+ yz 
25. a) WxZ wxy + wyz + wxyz b)xyz + Wyzt 
wxyz +wxyz+Wxyz e)yz+wxy + wr y+Wwryz 
d)wy+yz+xy+wxz+wxz 27. x(y+z) 


31L.x%Z+xz 33. Weuse induction onn. Ifn = 1, then we are 
looking at a line segment, labeled 0 at one end and | at the other 
end. The only possible value of k is also 1, and if the literal 
is x,, then the subcube we have is the 0-dimensional subcube 
consisting of the endpoint labeled 1, and if the literal is ¥;, then 
the subcube we have is the 0-dimensional subcube consisting 
of the endpoint labeled 0. Now assume that the statement is 
true for n; we must show that it is true for n + 1. If the lit- 
eral x,+1 (or its complement) is not part of the product, then 
by the inductive hypothesis, the product when viewed in the 
setting of n variables corresponds to an (n — k)-dimensional 
subcube of the n-dimensional cube, and the Cartesian product 
of that subcube with the line segment [0, 1] gives us a sub- 
cube one dimension higher in our given (n + 1)-dimensional 
cube, namely having dimension (n+ 1) —k, as desired. On the 
other hand, if the literal x,,,; (or its complement) is part of the 
product, then the product of the remaining k — 1 literals cor- 
responds to a subcube of dimension n — (k— 1) = (n+1)—k 
in the n-dimensional cube, and that slice, at either the 1-end 
or the 0-end in the last variable, is the desired subcube. 


Supplementary Exercises 


la)x=0, y=0,2=0;x=1, y=1,z=1 b)x=0, 
y=0,z=0;x =0,y = 0,z l;x =0, y 1, 2=:0; 
x ly=0,z 1: x l,y 1, 2= 02% l,y 1 
z=1c)Novalues 3.a) Yes b)Noc) Nod) Yes 5.22”. 


? 


Ta) If F(x1,...,%) = 1, then(F + G)(x11,...,%) = 
F(x1,.--,Xn) + G(x, ...,Xn) = 1 by the dominance law. 
Hence, F < F+G. b)If (FG)(%,...,x,) = 1, then 
F(x1,...,Xn)+ G(x], ..., %) = 1. Hence, F(x1,...,X%,) = 


1. It follows that FG < F. 9. Because F(x1,...,X,) = 1 
implies that F(x,,...,Xn) = 1, < is reflexive. Suppose that 
F < GandG < F.Then F(x1,...,x,) = 1 if and only 
if G(x, ...,%X,) = 1. This implies that F = G. Hence, 
< is antisymmetric. Suppose that F < G < H. Thenif 
F(x1,...,Xn) = I, it follows that G(x1,..., x.) = 1, which 
implies that H(x1,...,%,) = 1. Hence, F < H,so < 
is transitive. ll.a)x lLy 0, z 0 bx = 1 
y=0,z=0 dx=1,y=0,z=0 
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15. Yes, as atruth table shows 17. a)6 b)5 c)5 d)6 


19. x 
y 


21..x3 + x2x; 23. Suppose it were with weights a and b. 
Then there would be a real number T such that xa + yb > T 
for (1,0) and (0,1), but with xa + yb < T for (0,0) and (1,1). 
Hence,a > T,b > T,0 < T,anda+b < T. Thus,a 
and b are positive, which implies thata +b > a > T,a 
contradiction. 


CHAPTER 13 


Section 13.1 


1. a) sentence > noun phrase intransitive verb phrase 
= article adjective noun intransitive verb phrase => 
article adjective noun intransitive verb =... 
(after 3 steps) ...= the happy hare runs. 

b) sentence = noun phrase intransitive verb phrase 
= article adjective noun intransitive verb phrase 
= article adjective noun intransitive verb 
adverb...(after 4 steps)...= the sleepy tortoise runs 
quickly 

c) sentence => noun phrase __ transitive verb phrase 
noun phrase = article noun _ transitive verb phrase 
noun phrase = article noun transitive verb noun 
phrase = article noun transitive verb article 
noun =>... (after 4 steps)...= the tortoise passes the hare 
d) sentence = noun phrase __ transitive verb phrase 
noun phrase = article adjective noun transitive 
verb phrase noun phrase = article adjective noun 
transitive verb noun phrase = article adjective 
noun transitive verb article adjective noun 
=>... (after 6 steps)... the sleepy hare passes the happy 


tortoise 
3. The only way to get a noun, such as tortoise, at the end is to 


have a noun phrase at the end, which can be achieved only via 
the production sentence — noun phrase __ transitive verb 
phrase noun phrase. However, transitive verb phrase — 
transitive verb — passes, and this sentence does not contain 
passes. 

5.a) $3 1A>10B=> 101A > 1010B= 10101 b) Because 
of the productions in this grammar, every | must be followed 
by a 0 unless it occurs at the end of the string. c) All strings 
consisting of a 0 or a 1 followed by one or more repetitions of 
01 
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7 S = OSL = OOSI1 = O00S111 + 000111 
XaS = OS = OOS = OOSI =} OOSIL = 
00S111 = 0051111 001111 b) S>0S => 00S > 
001A + O011A > OO1LIIA S|} O01111 11.5 5 0SAB = 
OOSABAB = OOABAB = 00 AABB = OOIABB = 
0011BB > 00112B + 001122 13.a)S > 0,S > 1, 
S—+ 11 bS—> 18,5 >A oS > OAI,A — 1A, 
A —> 0A,A > id AS > OAA > IAAP A 
15. a) S > 00S, 5 > X b)S — 10A, A > 00A,A > 1 
c)S — AAS, S ~ BBS, AB > BA, BA > AB,S— 4, 
A—>0,B-— 1 d)S — OO000000000A, A > 0A, A > 
e)S > AS, S — ABS, S — A, AB > BA, BA = AB, 
A->0,B—> 1 f)S — ABS,S > 41, AB —> BA, 
BA > AB,A > 0,B > 1 g)S > ABS,S => T, 
S—+U,T > AT,T > A,U > BU,U > B,AB > BA, 
BA > AB,A>O0,B->1 17.aS > 08,S > i 
b)S > AO, A > 1A,A > 2 ©) S > 0005S, 5 > dX 
19. a) Type 2, not type 3. b) Type 3. c) Type 0, not type 1 
d) Type 2, not type 3. e) Type 2, not type 3 f) Type 0, not 
type 1 g) Type3 h) Type 0, not type 1 i) Type 2, not type 3 
j) Type 2, nottype 321. Let S$; and S> be the start symbols 
of G; and Go, respectively. Let S be a new start symbol. 
a) Add S and productions S + S$; and S — S». b) Add S$ 
and production S — $,S2. c) Add S and production S > X 
and S$ > $\S. 


23. a) sentence 


noun phrase intransitive verb phrase 


JIN 


article adjective noun intransitive verb 


the happy hare runs 
: = ; a 
noun phrase intransitive verb phrase noun phrase transitive verb phrase noun phrase 
J \ intransitive s\ A ~*~ / \ 
article adjective noun verb adverb article noun transitive verb article noun 


the sleepy __ tortoise runs quickly the tortoise passes the hare 


d) sentence 


noun phrase transitive verb phrase 


LIN 


article adjective noun transitive verb article 


the sleepy hare passes the 


25.a) Yes b) No c) Yes d) No 


signed integer 


Y/N 


sign inte, ger 


- digit integer 


digit integer 


0 digit 


29.a) S — (sign) (integer) 

S — (sign) (integer) . (positive integer) 

(sign) > + 

(sign) > 

(integer) > (digit) 

(integer) — (integer) (digit) 
(digit) > i,i = 1,2,3,4,5, 6,7, 8,9,0 
(positive integer) — (integer) (nonzero digit) 
( 
( 


positive integer) — (nonzero digit) (integer) 


( 
positive integer) —> ( 
(integer) 

(positive integer) — (nonzero digit) 
(nonzero digit) > i,i = 1,2,3,4,5, 6,7, 8,9 
b) (signed decimal number) ::= (sign) (integer) | 
(sign) (integer) . (positive integer) 
(sign) = +|— 
(integer) ::= (digit) | (integer) (digit) 
(digit) :=0|1|2|3|4|5|6|7|8 |9 


integer) (nonzero digit) 
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noun phrase 


JIN 


adjective noun 


happy tortoise 


(nonzero digit) := 1|2|3|4|5|6|7|8|9 
(positive integer) ::= (integer) (nonzero digit) | 
(nonzero digit) (integer) | (integer) 
(nonzero integer) (integer) | (nonzero digit) 


c) signed decimal number 


ie 


sign integer positive integer 


Le 


— integer digit nonzero digit 


digit 1 4 


3 
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31. 


a) (identifier) ::= (Icletter) | (identifier) (Icletter) 
(Icletter) :=a|b|c|--+-|z 


b) (identifier) ::= (Icletter) (Icletter) (Icletter) | (Icletter) (Icletter) (Icletter) (Icletter) | 
(Icletter) (Icletter) (Icletter) (Icletter) (Icletter) | 
(Icletter) (Icletter) (Icletter) (Icletter) (Icletter) (Icletter) 
(Icletter) :=a|b|\c|---|z 
c) (identifier) ::= (ucletter) | (ucletter) (letter) | (ucletter) (letter) (letter) | 
(ucletter) (letter) (letter) (letter) | (ucletter) (letter) (letter) (letter) (letter) | 
(ucletter) (letter) (letter) (letter) (letter) (letter) 
(letter) ::= (Icletter) | (ucletter) 
(Icletter) :=a|b|c|---|z 
(ucletter) ::= A|B\|C|---|Z 
d) (identifier) ::= (Icletter) (digitorus) (alphanumeric) (alphanumeric) (alphanumeric) | 
(Icletter) (digitorus) (alphanumeric) (alphanumeric) (alphanumeric) (alphanumeric) 
(digitorus) ::= (digit) | _ 
(alphanumeric) ::= (letter) | (digit) 
(letter) ::= (Icletter) | (ucletter) 
(Icletter) :=a|b|c|---|z 
(ucletter) := A| B|C|---|Z 
(digit) :=0|1|]2|---|9 
33. (identifier) ::= (letterorus) | (identifier) (symbol) 
(letterorus) ::= (letter) | _ 
(symbol) ::= (letterorus) | (digit) 
(letter) ::= (Icletter) | (ucletter) 
(Icletter) :=a|b|c|---|z 
(ucletter) ::= A|B|C |---| Z 
(digit) :=0|1|2|---|9 
35. numeral ::= sign? nonzerodigit digit decimal? | sign? 0 decimal? 
sign w= + |—- 
nonzerodigit ::= 1|2|---|9 
digit ::= 0 | nonzerodigit 
decimal ::= .digit* 
37. identifier ::= letterorus symbolx 
letterorus ::= letter | _ 
symbol ::= letterorus | digit 
letter ::= Icletter | ucletter 
Icletter::=a\b\c|---|z 
ucletter ::= A| B|C|---|Z 
digit :=0|1|2|---|9 
39. a) (expression) 
(term) (term) (addOperator) 
(factor) (factor) (factor) (mulOperator) (addOperator) 
(identifier) (identifier) (identifier) (mulOperator) (addOperator) 
abcx+ 
b) Not generated 
c) (expression) 
(term) 


(factor) (factor) (mulOperator) 

(expression) (factor) (mulOperator) 

(term) (term) (addOperator) (factor) (mulOperator) 

(factor) (factor) (addOperator) (factor) (mulOperator) 
(identifier) (identifier) (addOperator) (identifier) (mulOperator) 
xy—-Z* 
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d) (expression) 
(term) 
(factor) (factor) (mulOperator) 

(factor) (expression) (mulOperator) 

(factor) (term) (mulOperator) 

(factor) (factor) (factor) (mulOperator) (mulOperator) 


(factor) (factor) (expression) (mulOperator) (mulOperator) 


woe Re a a a 


(factor) (factor) (term) (term) (addOperator) (mulOperator) (mulOperator) 
(factor) (factor) (factor) (factor) (addOperator) (mulOperator) (mulOperator) 
(identifier) (identifier) (identifier) (identifier) (addOperator) (mulOperator) (mulOperator) 
wxyz—x*/ 

e) (expression) 

(term) 

(factor) (factor) (mulOperator) 

(factor) (expression) (mulOperator) 

(factor) (term) (term) (addOperator) (mulOperator) 

(factor) (factor) (factor) (addOperator) (mulOperator) 
(identifier) (identifier) (identifier) (addOperator) (mulOperator) 


ade-—*x 


41. a) Not generated 
b) (expression) 
(term) (addOperator) (term) 
(factor) (mulOperator) (factor) (addOperator) (factor) (mulOperator) (factor) 
(identifier) (mulOperator) (identifier) (addOperator) (identifier) (mulOperator) (identifier) 
a/b+c/d 
c) (expression) 
(term) 
(factor) (mulOperator) (factor) 


)( 
(factor) (mulOperator) ({expression) ) 
)( 


)¢ 
(factor) (mulOperator) ((term) (addOperator) (term) ) 
(factor) (mulOperator) ((factor) (addOperator) (factor) ) 
(identifier) (mulOperator) ((identifier) (addOperator) (identifier) ) 
mx (n+ Pp) 
d) Not generated 
e) (expression) 
(term) 
(factor) (mulOperator) (factor) 
((expression) )(mulOperator) ((expression) ) 
((term) (addOperator) (term) )(mulOperator) ((term) (addOperator) (term))  ((factor)(addOperator) (factor) ) (mulOperator) ((factor) (addOp 
((identifier) (addOperator) (identifier) )(mulOperator) ((identifier) (addOperator) (identifier) ) 


(m +n) * (p — q) 
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Section 13.2 


3.a) 01010 b)01000 c) 11011 5.a) 1100 b) 00110110 c) 11111111111 


We 25,0 25,0 25,9 25,10 25,15 25,20 


x E{G,2,9} 


@ =cola 
& =root beer 
g = ginger ale 


v= Valid ID a="Enter user ID" 
i= Invalid ID b ="Enter password" 
p = Valid password c = Prompt 
q=Invalid password x= Any input 


13. {5, 10, 25}, open 


10, nothing 10, nothing 


10, nothing 


Start 


5, nothing 5, nothing 


5, nothing 


5, nothing 


25, open 


10, open 


25, open 


15. Let sg be the start state and let s; be the state representing 
a successful call. From so, inputs of 2, 3, 4, 5, 6, 7, or 8 send 
the machine back to sp with output of an error message for the 
user. From so an input of 0 sends the machine to state s;, with 
the output being that the 0 is sent to the network. From so an 
input of 9 sends the machine to state s2 with no output; from 
there an input of 1 sends the machine to state s3 with no out- 
put; from there an input of 1 sends the machine to state 5; with 
the output being that the 911 is sent to the network. All other 
inputs while in states s2 or 53 send the machine back to sg with 
output of an error message for the user. From sg an input of 1 
sends the machine to state s4 with no output; from s4 an input 
of 2 sends the machine to state s5 with no output; and this path 
continues in a similar manner to the 911 path, looking next 
for 1, then 2, then any seven digits, at which point the machine 
goes to state s; with the output being that the ten-digit input 
is sent to the network. Any “incorrect” input while in states s5 
or s6 (that is, anything except a 1 while in s5 or a 2 while in 
s¢) sends the machine back to sg with output of an error mes- 
sage for the user. Similarly, from s4 an input of 8 followed by 
appropriate successors drives us eventually to s;, but inappro- 
priate outputs drive us back to sg with an error message. Also, 
inputs while in state s4 other than 2 or 8 send the machine 
back to state so with output of an error message for the user. 


17. 1,0 
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19. R,1 


x—{O},0 


L-{T},0 


23.a) 11111 
b) 1000000 
c) 100011001100 


Section 13.3 


1. a) {000,001, 1100, 1101} b) {000,0011,010,0111} 
c) (00, O11, 110, 1111}  d) {000000, 000001, 000100, 
000101, 010000, 010001, 010100, 010101} 3. A={1, 101}, 
B = {0,11,000}; A = {10,111,1010, 1000, 10111, 101000}, 
B = {A}; A = {A, 10}, B = {10, 111, 1000} or A = {A}, 
B {10, 111, 1010, 1000, 10111, 101000} 5. a) The set 
of strings consisting of zero or more consecutive bit pairs 10 
b) The set of strings consisting of all 1s such that the number 
of 1s is divisible by 3, including the null string c) The set of 
strings in which every | is immediately preceded by a0 d) The 
set of strings that begin and end with a 1 and have at least two 
1s between every pair of Os__7. A string is in A* if and only 
if it is a concatenation of an arbitrary number of strings in A. 
Because each string in A is also in B, it follows that a string in 
A* is also a concatenation of strings in B. Hence, A* C B*. 
9.a) Yes b)Yes c)No d)No e)Yes f)Yes_ 11. a) Yes 
b) No c) Yes d) No 13. a) Yes b) Yes c) No d) No e) No 
f) No 15. We use structural induction on the input string y. 
The basis step considers y = A, and for the inductive step we 
write y = wa, where w € I* anda € I. For the basis step, we 
have xy = x, so we must show that f(s, x) = f(f(s, x), A). 
But part (i) of the definition of the extended transition func- 
tion says that this is true. We then assume the inductive 


D— {E},0 


L—{R},0 


hypothesis that the equation holds for w and prove that 
f(s, xwa) = f (f(s, x), wa). By part (ii) of the definition, 
the left-hand side of this equation equals f(f(s, xw), a). 
By the inductive hypothesis, f(s, xw) = f(f(s, x), w), 
so f(f(s, xw),a) = f(f(f(s, x), w), a). The right- 
hand side of our desired equality is, by part (i) of the 
definition, also equal to f(f(f(s, x), w), a), as desired. 
17. {0, 10, 11}{0, 1}* 19.{0"1" | m => Oandn = 1} 
21. {A}U {O} {1} * {0} U (10, 11}{0, 1} * U {O}{1} * {01} {0, 1}* U 
{O}{1}*{OO}{O}*{1}{0, 1}* 23. Let so be the only final state, 
and put transitions from s2 to itself on either input. Put a tran- 
sition from the start state sq to s; on input 0, and a transition 
from s; to s2 on input 1. Create state s3, and have the other 
transitions from sp and s; (as well as both transitions from 33) 
lead to 53. 25. Start state so, only final state s3; transitions 
from so to sg on 0, from sg to sy on 1, from s; to sz on 0, 
from s; to s; on 1, from sz to sg on O, from sz to s3 on 1, 
from s3 to s3 on 0, from s3 to s3 on 1 27. Have five states, 
with only s3 final. Fori = 0, 1, 2, 3, transition from s; to 
itself on input | and to 5;4; on input 0. Both transitions from 
s4 are to itself. 29. Have four states, with only s3 final. For 
i = 0, 1, 2, transition from s; to sj) on input 1 but back to 
Sq on input 0. Both transitions from s3 are to itself. 


33. Start state so, only final state s;; transitions from so to so 
on 1, from sg to s; on 0, from s; to sy; on 1; from s; to so on O 


37. Suppose that such a machine exists, with start state sq 
and other state s;. Because the empty string is not in the 
language but some strings are accepted, we must have s; 
as the only final state, with at least one transition from so 
to s;. Because the string 0 is not in the language, the tran- 
sition from so on input 0 must be to itself, so the transi- 
tion from so on input 1 must be to s;. But this contradicts 
the fact that 1 is not in the language. 39. Change each fi- 
nal state to a nonfinal state and vice versa. 41. Same ma- 
chine as in Exercise 25, but with so, sj, and s2 as the final 
states 43. {0,01,11} 45. {A,0} U {O" 1" |m>1,n > 1} 
47. {10" |n > O} U {10"10" | n,m > 0} 49. The union of 
the set of all strings that start with a 0 and the set of all strings 
that have no Os 
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53. Add a nonfinal state s3 with transitions to s3 from sp on 
input 0, from sj on input 1, and from s3 on input 0 or 1. 


55. a) Start 


c) 


Start 


0,1 


57. Suppose that M is a finite-state automaton that accepts 
the set of bit strings containing an equal number of Os and Is. 
Suppose M has n states. Consider the string 0”+!1”"*!. By the 
pigeonhole principle, as M processes this string, it must en- 
counter the same state more than once as it reads the first + 1 
Os; so let s be a state it hits at least twice. Then k Os in the input 
takes M from state s back to itself for some positive integer 
k. But then M ends up exactly at the same place after reading 
ortl+k n+l og it will after reading 0"+!1”+!. Therefore, be- 
cause M accepts 0"+! 1”! it also accepts 0" +44! 1"+!, which 
is acontradiction. 59. We know from Exercise 58d that the 
equivalence classes of Rx are a refinement of the equivalence 
classes of Rx_; for each positive integer k. The equivalence 
classes are finite sets, and finite sets cannot be refined indef- 
initely (the most refined they can be is for each equivalence 
class to contain just one state). Therefore this sequence of re- 
finements must remain unchanged from some point onward. 
It remains to show that as soon as we have R, = Ryj+1, then 
Ry = Rm for all m > n, from which it follows that R, = Rx, 
and so the equivalence classes for these two relations will be 
the same. By induction, it suffices to show that if R, = Rn+t, 
then Rn+1 = Rn+2. Suppose that Rn41 ~ Rn+2. This means 
that there are states s and ¢ that are (n + 1)-equivalent but not 
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(n + 2)-equivalent. Thus there is a string x of length n + 2 
such that, say, f(s, x) is final but f(t, x) is nonfinal. Write 
x = aw, wherea € I. Then f(s, a) and f(t, a) are not 
(n + 1)-equivalent, because w drives the first to a final state 
and the second to a nonfinal state. But f(s, a) and f(t, a) 
are n-equivalent, because s and t are (n + 1)-equivalent. This 
contradicts the fact that Ry, = Rn+1. 61. a) By the way the 
machine M was constructed, a string will drive M from the 
start state to a final state if and only if that string drives M from 
the start state to a final state. b) For a proof of this theorem, see 
a source such as Introduction to Automata Theory, Languages, 
and Computation (2nd Edition) by John E. Hopcroft, Rajeev 
Motwani, and Jeffrey D. Ullman (Addison-Wesley, 2000). 


Section 13.4 


1. a) Any number of 1s followed by a0 b) Any number of 
1s followed by one or more Os_c) 111 or 001 d)A string 
of any number of Is or 00s or some of each ina row e)A 
or a string that ends with a | and has one or more Os be- 
fore each 1 f) A string of length at least 3 that ends with 00 
3.a)No b)No c) Yes d) Yes e) Yes f) No g) No h) Yes 
5.a)0U11U010 b) 000000* c) OU1l(OUNO@U1)* 
d) 0*10* e) (1U01U001)* 7. a) 00*1 b) OUINOU1OU 
1)*0000* c) O*1* U1*0* d) 11(111)*(00)* 9. a) Have the 
start state sq, nonfinal, with no transitions. b) Have the start 
state so, final, with no transitions. c) Have the nonfinal start 
state sq and a final state s; and the transition from so to s1 
on input a. 11. Use an inductive proof. If the regular ex- 
pression for A is %, A, or x, the result is trivial. Otherwise, 
suppose the regular expression for A is BC. Then A = BC 
where B is the set generated by B and C is the set generated by 
C. By the inductive hypothesis there are regular expressions 
B’ and C’ that generate B® and C¥, respectively. Because 
A® = (BC) = C¥ BR, CB’ isa regular expression for A®. 
If the regular expression for A is B U C, then the regular ex- 
pression for A¥ is B’ UC’ because (BUC)® = (B®) U(CR). 
Finally, if the regular expression for A is B*, then it is easy to 
see that (B’)* is a regular expression for A®. 


15.5—> 0A, S> 1B, S-0, A> 0B, A> 1B,B > 
0OB,B—> 1B 17.8 > 0C,S > 1A,S > 1,A > 1A, 
A> 0C,A—>1,B — 0B8,B > 1B,B ~0,B > 1, 
C > 0C,C > 1B,C = 1. 19. This follows because 
input that leads to a final state in the automaton corresponds 
uniquely to a derivation in the grammar. 21. The “only if” 
part is clear because / is finite. For the “if” part let the states be 
Sigs Sijs Sins +++5 Si,, Where n = I(x). Because n > |S|, some 
state is repeated by the pigeonhole principle. Let y be the part 
of x that causes the loop, so that x = uyv and y sends s; to 
s;, for some j. Then uy*y € L(M) for all k. Hence, L(M) 
is infinite. 23. Suppose that L = 02" 1% ,.n- = 0,.,2°.,4 
were regular. Let S be the set of states of a finite-state machine 
recognizing this set. Let z = 07” 1" where 3n > |S|. Then 
by the pumping lemma, z = 0771" = uw, I(v) > 1, and 
uviw € {02"1" | n > O}. Obviously v cannot contain both 
0 and 1, because v2 would then contain 10. So v is all Os or 
all 1s, and hence, uv-w contains too many Os or too many 
ls, so it is not in L. This contradiction shows that L is not 
regular. 25. Suppose that the set of palindromes over {0, 1} 
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were regular. Let S be the set of states of a finite-state machine 
recognizing this set. Let z = 0"10”, where n > |.S|. Apply the 
pumping lemma to get uv'w € L for all nonnegative integers 
i where /(v) > 1, and/(uv) < |S|, and z = 0"10" = www. 
Then v must be a string of Os (because n > ||), so uv-w is 
not a palindrome. Hence, the set of palindromes is not regu- 
lar. 27. Let z = 1; then 111 ¢ L but 101 € L, so 11 and 
10 are distinguishable. For the second question, the only way 
for lz to be in L is for z to end with O1, and that is also the 
only way for 11z to be in L, so 1 and 11 are indistinguishable. 
29. This follows immediately from Exercise 28, because the n 
distinguishable strings must drive the machine from the start 
state ton different states. 31. Any two distinct strings of the 
same length are distinguishable with respect to the language 
P of all palindromes, because if x and y are distinct strings 
of length n, then xx® € P but yx® ¢ P. Because there 
are 2” different strings of length n, Exercise 29 tells us that 
any deterministic finite-state automaton for recognizing palin- 
dromes must have at least 2” states. Because n is arbitrary, this 
is impossible. 
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Section 13.5 


1. a) The nonblank portion of the tape contains the string 1111 
when the machine halts. b) The nonblank portion of the tape 
contains the string 011 when the machine halts. c) The non- 
blank portion of the tape contains the string 00001 when the 
machine halts. d) The nonblank portion of the tape contains 
the string 00 when the machine halts. 3. a) The machine 
halts (and accepts) at the blank following the input, having 
changed the tape from 11 to 01. b) The machine changes ev- 
ery other occurrence of a 1, if any, starting with the first, to 
a 0, and otherwise leaves the string unchanged; it halts (and 
accepts) when it comes to the end of the string. 5. a) Halts 
with 01 on the tape, and does not accept b) The first 1 (if any) 
is changed to a 0 and the others are left alone. The input is not 
accepted. 

7. (so, 0, 51, 1, R), (50, 1, 50, 1, R) 9. (80, 0, 50, 0, R), 
(so, 1, 51, 1, R), (51, 51, 0, R), (51, 1, 51,0, R) 11. (so, 0, 
51,0, R), (so, 1, so, 0, R), (51, 0, 51, 0, R), (51, 1, 50, 0, R), 
(s1, B, so, B, R) 13. (so, 0, so, 0, R), (50, 1, 51, 1, R), 
(51, 0, 51,0, R), (S1, 1, so, 1, R), (so, B, 52, B, R) 15. 1f 
the input string is blank or starts with a 1 the machine halts in 
nonfinal state sg. Otherwise, the initial 0 is changed to an M 
and the machine skips past all the intervening Os and Is until 
it either comes to the end of the input string or else comes to 
an M. At this point, it backs up one square and enters state s>. 
Because the acceptable strings must have a | at the right for 
each 0 at the left, there must be a | here if the string is accept- 
able. Therefore, the only transition out of s2 occurs when this 
square contains a 1. If it does, the machine replaces it with 
an M and makes its way back to the left; if it does not, the 
machine halts in nonfinal state sz. On its way back, it stays in 
53 as long as it sees Is, then stays in s4 as long as it sees Os. 
Eventually either it encouters a 1 while in s4 at which point it 
halts without accepting or else it reaches the rightmost M that 
had been written over a 0 at the start of the string. If it is in 
53 when this happens, then there are no more Os in the string, 
so it had better be the case that there are no more Is either; 
this is accomplished by the transitions (s3, M, 55, M, R) and 
(s5, M, 56, M, R), and s¢ is a final state. Otherwise the ma- 
chine halts in nonfinal state s5. If it is in sy when this M is 
encountered, things start all over again, except now the string 
will have had its leftmost remaining 0 and its rightmost re- 
maining | replaced by Ms. So the machine moves, staying in 
state s4, to the leftmost remaining 0 and goes back into state 
sg to repeat the process. 


17. (so, B, so, B, L), (so, 0, 51, 0, L), (81, B, 52, E, R), 
(s2, M, so, M, R), (s2, 0, 53, M, R), (53, 0, 53, 0, R), 
(s3, M, s3, M, R), (53, 1, sa, M, R), (54, 1, s4, 1, R), 
(s4, M, sa, M, R), (s4, 2, 55, M, R), (55, 2, 55, 2, R), 
(s5, B, s6, B, L), (s6, M, sg, M, L), (56, 2, 57, 2, L), 
(s7, 0, S7, 0, L), (s7, 1, S7, 1 EL); (s7, 2, S7, 2, L), 
(s7, M, s7, M, L), (s7, E, 82, E, R), (sg, M, sg, M, L), 
(sg, E, 59, E, L) where M and E are markers, with FE marking 
the left end of the input 


19. (so, 1, 51, B, R), (81, 1, 52, B, R), (s2, 1, 53, B, R), 
(s3, 1, s4, 1, R), (s1, B, sa, 1, R), (82, B, sa, 1, R), 
(s3, B, 54, 1, R) 

21. (so, 1, 51, B, R), (51, 1, 2, B, R), (s1, B, 56, B, R), 
(so, 1, 53, B, R), (so, B, 56, B, R), (53, 1, s4, B, R), 
(s3, B, 56, B, R), (s4, 1, 55, B, R), (54, B, 56, B, R), 
(s6, B, sig, 1, R), (55, 1, 55, B, R), (85, B, 57, 1, R), 
(s7, B, sg, 1, R), (sg, B, 89, 1, R), (89, B, sto, 1, R) 

23. (so, 1, so, 1, R), (89, B, 51, B, L), (81, 1, 52, 0, L), 
(s2, 0, s2, 0, L), (s2, 1, 53, 0, R), (s2, B, s6, B, R), 
(s3, 0, 53, O, R), (53, 1, 53, 1, R), (83, B, 54, 1, R), 
(s4, B, s5, 1, L), (55, 1, 55, 1, L), (85, 0, 52, 0, L), 
(56, 0, 56, 1, R), (56, 1, 57, 1, R), (86, B, 57, B, R) 

25. (so, 0, so, 0, R), (50, *, 55, B, R), (53, *, 53, *, L), 
(s3, 0, 53, 0, L), (53, 1, 53, 1, L), (83, B, so, B, R), 
(s5, 1, 55, B, R), (85, 0, 55, B, R), (55, B, 56, B, L), 
(se; B; 86, By, L); (6; 0; 875. 1, Ds. 75 0, 875. 1, B); 
(so, 1, 53, 0, R), (51, 1, 51, 1, R), (51, *, 52, *, R), 
(s2, 0, s2, 0, R), (52, 1, 53, 0, L), (52, B, s4, B, L), 
(s4, 0, sa, 1, L), (sa, *, 5g, B, L), (sg, 0, sg, B, L), 
(sg, 1, sg, B, L) 

27. Suppose that s,, is the only halt state for the Turing ma- 
chine in Exercise 22, where m is the largest state number, 
and suppose that we have designed that machine so that when 
the machine halts the tape head is reading the leftmost 1 of 
the answer. Renumber each state in the machine for Exer- 
cise 18 by adding m to each subscript, and take the union of 
the two sets of five-tuples. 29. a) No b) Yes c) Yes d) Yes 
31. (so, B, 51, 1, L), (so, 1, 51, 1, R), (81, B, So, 1, R) 


Supplementary Exercises 


l.a)S > 00S111,5 ~ 24 b)S — AABS, AB > BA, 
BA> AB,A>0,B>1,S 7100S > ET,T > 
OTA,T — 1TB,T > i4,0A > AO, 1A > A1,0B > BO, 
1B—> B1,EA—> E0I,EB—> E1lE>d 


3 S | S 
A Pac A 
JN | ZINN 
( A) / ~ | ) ( A) 
la -_ JIN 
J a 
| Pa \, 
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7. No, take A = {1, 10} and B = {0,00}. 9. No, take A = 
{00, 000, 00000} and B = {00, O00}. Il.a)1l b)1 c)2 
d)3 e)2 f)4 


23. Construct the deterministic finite automaton for A with 
states S and final states F. For A use the same automaton but 
with final states S§ — F. 


25.) Start 
a 


17. a) ntktlyynk b) nrkt+lyyn 
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27. Suppose that L = {1? | p is prime} is regular, and let S be 
the set of states in a finite-state automaton recognizing L. Let 
z = 1” where p is a prime with p > |S| (such a prime exists 
because there are infinitely many primes). By the pumping 
lemma it must be possible to write z = uvw with /(uv) < |S|, 
l(v) > 1, and for all nonnegative integers 7, uvw é€ L. 
Because z is a string of all Is, u = 1%, v = 1°, andw = 1°, 
wherea+b+c= p,a+b <n, and b > 1. This means that 
uviw — 14126 — p@tbte+b-1) — 1p+bG-)_ Now take 
i = p+1.Thenuv'w = 1?"+) Because p(1+5) is not prime, 


uv'w ¢ L, which is a contradiction. 29. (so, *, 55, B, L), 
(so, 0, so, O, R), (50, 1, 51, 0, R), (51, *, 52, *, R), 
(sj, 1, 51, 1, R), (s2, 0, 52, 0, R), (2, 1, 53, 0, L), (52, B, 4, 
B, L), (83, *, 83, *, L), (53, 0, 53, 0, L), (83, 1, 53, 1, L), 
(s3, B, so, B, R), (s4, *, 88, B, L), (sa, 0, 54, B, L), (85, 0, 
55, B, L), (55, B, so, B, R), (s6, 0, 57, 1, R), 
(s6, B, s6, B, R), (57, 0, 57, 1, R), (57,1,57, 1, R), 
(sg, 0, 5g, 1, L), (sg, 1, 58,1,L) 


APPENDIXES 


Appendix 1 


1. Suppose that 1’ is also a multiplicative identity for the real 
numbers. Then, by definition, we have both 1 - 1’ = 1 and 
1-1'’=1',sol‘=1. 3. For the first part, it suffices to show 
that [(—x) - y] + (x - y) = 0, because Theorem 2 guarantees 
that additive inverses are unique. Thus [(—x)- y] + (x- y) = 
(—x + x) - y (by the distributive law) = 0 - y (by the inverse 
law) = y - 0 (by the commutative law) = 0 (by Theorem 5). 
The second part is almost identical. 5. It suffices to show 
that [(—x) - (~y)] + [-@ - y)] = 0, because Theorem 2 
guarantees that additive inverses are unique: [(—x) - (—y)]+ 
[—( - y)] = [(—x) - yy] + [Cx - y] (by Exercise 3) 
= (—x) - [((—y) + y] (by the distributive law) = (—x) - 0 
(by the inverse law) = 0 (by Theorem 5). 7. By definition, 
—(—x) is the additive inverse of —x. But —x is the additive 
inverse of x, so x is the additive inverse of —x. Therefore 
—(-—x) = x by Theorem 2. 9. It suffices to show that 
(—x — y) + (x + y) = 0, because Theorem 2 guarantees 
that additive inverses are unique: (-x — y) + (x + y) = 
[(—x) + (—y)] + (+ y) (by definition of subtraction) = 
[(—-y) + (—x)] + (& + y) (by the commutative law) = 
(—y) + [(—x) + (x + y)] (by the associative law) = (—y) + 
[(—x + x) + y] (by the associative law) = (—y)+ (0+ y) 
(by the inverse law) = (—y) + y (by the identity law) = 
0 (by the inverse law). 11. By definition of division and 
uniqueness of multiplicative inverses (Theorem 4) it suffices 
to prove that [(w/x) + (y/z)]- («@-z) =w-z+x-y. 
But this follows after several steps, using the distributive law, 
the associative and commutative laws for multiplication, and 
the definition that division is the same as multiplication by 
the inverse. 13. We must show that if x > 0 and y > 0, 
then x - y > 0. By the multiplicative compatibility law, the 
commutative law, and Theorem 5, we have x-y > 0-y = 0. 
15. First note that if z < 0, then —z > 0 (add —z to both sides 
of the hypothesis). Now given x > y and —z > 0, we have 
x + (—z) > y- (—z) by the multiplicative compatibility law. 
But by Exercise 3 this is equivalent to —(x - z) > —(y- Z). 
Then add x - z and y - z to both sides and apply the various 
laws in the obvious ways to yield x -z < y-z. 17. The 
additive compatibility law tells us that w+ y < x + y and (to- 
gether with the commutative law) that x + y < x + z. By the 
transitivity law, this gives the desired conclusion. 19. By 
Theorem 8, applied to 1/x in place of x, there is an inte- 
ger n (necessarily positive, because 1/x is positive) such that 
n > 1/x. By the multiplicative compatibility law, this means 
thatn-x >1. 21. We must show that if (w, x) ~ (w’, x’) and 
(y, 2) ~ (’, 2’), then (w+ y,x +z) ~ (w+ y', x’ +2’) and 
that (w-y+x-z,x-yt+w-z) ~ (Wy $x! +2), x! y’+w" 2’). 
Thus we are given that w+x’ = x-+w’ and that y+z’ =z+y’, 
and we want to show that w+ y+x/+z’ =x+z+w’+y’ and 
that w-y+x-ztx!-y'+w'z! = x-y+w-z+w'-y’+x'+z'. For the 
first of the desired conclusions, add the two given equations. 


Answers to Odd-Numbered Exercises S-97 


For the second, rewrite the given equations as w—x = w’ —x’ Appendix 3 

and y — z= y’ — z’, multiply them, and do the algebra. TT 

1. After the first block is executed, a has been assigned the 
original value of b and b has been assigned the original value 
of c, whereas after the second block is executed, b is assigned 
the original value of c and a the original value of c as well. 
3. The following construction does the same thing. 


Appendix 2 
l.a) 2? b) 2° ¢)2* 3.a)2y b)2y/3 c) y/2 


i := initial value 
3: while i < final value 
statement 
i:=i+l1 
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Algebra 
fundamental theorem of, 241 
origin of term, 192 
Algebra, Boolean 
abstract definition of, 817 
definition of, 843 
identities of, 814-816 
ALGOL, 854 
Algorithm(s), 191-202, 218-229, 232, 
div and mod, 253 
addition, 250 
approximation, 235, 715 
binary search, 194 
binary search tree, 757-769 
Boolean product of zero—one matrices, 223 
brute-force, 224 
brute-force for closest pair of points, 225 
coloring graphs, 731, 734 
complexity of, 218-229 
average-case, 482-484 
of breadth-first search algorithms, 791 
of depth-first search algorithms, 789 
of Dijkstra’s algorithm, 714 
of merge sort, 532 
computer time used by, 228 
constructing base b expansions, 248 
correctness of, 193 
deferred acceptance, 204, 343 
definition of, 191 
definiteness of, 193 
describing using English, 192 
Dijkstra’s, 709-714, 737 
divide-and-conquer, 527, 565 
division, 239, 306 
division, 239 
effectiveness of, 193 
Euclidean, 267, 347 
Euler circuits, 696-697 
Euler paths, 697-698 
evaluating polynomials, 230 
extended Euclidean, 270, 273 
fast integer multiplication, 528-529 
fast matrix multiplication, 528 
Fibonacci numbers, 365-367 
finding maximum element, 193 
finiteness of, 193 
Fleury’s, 697, 706 
Floyd’s, 717 
Gale-Shapley, 204 
generality of, 193 
generating combinations, 437-438 
generating permutations, 434-436 
greedy, 198, 232, 324-325, 764, 798, 804 
greedy change-making, 199 
history of word, 192 
Huffman coding, 763-764 
inorder traversal, 779 
input to an, 193 
iterative, for Fibonacci numbers, 366 
Kruskal’s, 799-801, 804 
linear search, 194, 232 
matrix multiplication, 222 
merge sort, 367-370 
minimum spanning trees, 798-802 
modular exponentiation, 253 
Monte Carlo, 463-465 


multiplication, 251 
optimal, 230 
origin of term, 246 
output of an, 193 
parallel, 661 
postorder traversal, 779 
preorder traversal, 777 
Prim’s, 798-799, 804 
probabilistic, 445, 463-465, 494 
properties of, 193 
proving primality, 262 
pseudocode, 192-193 
reverse delete, 803 
recursive, 360-370, 378 
binary search, 363 
correctness of, 364-367 
for computing a”, 361 
for computing greatest common divisor, 362 
for Fibonacci numbers, 365 
linear search, 363 
modular exponentiation, 363 
searching, 194-196, 232 
binary, 528 
breadth-first, 789-791, 804 
depth-first, 787-789, 804 
linear average-case complexity of, 483-484 
linear recursive sequential, 363 
sequential search, 194 
serial, 661 
shortest-path, 709-714 
Sollin’s, 803 
sorting, 196-198 
topological, 627-629, 634 
spanning trees, 798-802 
specifying, 192 
ternary search, 203 
topological sorting, 628 
transitive closure, 603 
traversal, 773-779 
Warshall’s, 603-606, 634 
Algorithm for finding maximum 
time complexity of, 219 
Algorithmic paradigm, 224, 232 
backtracking, 224 
brute force, 224 
divide-and-conquer, 224 
dynamic programming, 224, 507 
greedy algorithms, 198, 224 
probabilistic algorithms, 224, CH7 
Alice, 302, 303, 305 
Alice in Wonderland, 50 
Alpha-beta pruning, 769, 809 
Alphabet, 849, 899 
Alphanumeric character, 391 
A Mathematician’s Apology, 97 
Ambiguous context-free grammar, 901 
Amino acids, 389 
Analytic Engine, 31 
Ancestors of vertex, 747, 803 
AND, 18 
AND gate, 21, 823, 843 
Antecedent of a conditional statement, 6 
Antichain, 637 


Antisymmetric relation, 577-578, 633 
representing 
using digraphs, 595 
using matrices, 591-592 
Appel, Kenneth, 728 
Application(s) 
of backtracking, 791-793 
Bayes’ theorem, 471-472 
congruences, 287 
graphs colorings, 731-732 
graph theory, 644-649, 658-663, 680-681, 687, 
698, 702, 731-732 
inclusion—exclusion, 558-564 
pigeonhole principle, 403-405 
propositional logic, 16-22 
satisfiability, 32-34 
of trees, 757-769 
Approximation algorithm, 235, 715 
Arborescence, 807 
Archimedean property, A-5 
Archimedes, A-4 
Archimedes’ screw, A-4 
Argument, 69, 110 
Cantor diagonalization, 173, 186 
premises of, 69 
propositional logic, 70 
valid, 69, 110 
Argument form, 70, 110 
valid, 70, 110 
Aristotle, 2, 3 
Arithmetic 
fundamental theorem of, 306, 336-337 
modular, 240-306 
modulo m, 243 
Arithmetic mean, 100, 332 
Arithmeticorum Libri Duo (Maurolico), 313 
Arithmetic progression, 186 
of primes, 262, 263 
Array linear, 662 
two-dimensional, 662 
Arrow, Peirce, 36 
Ars Conjectandi (Bernoulli), 459 
Art Gallery 
problem, 735 
theorem, 735 
Article, 848 
Articulation points, 683 
Artificial intelligence, 138, 380 
fuzzy logic in, 16 
Assertion 
final, 372, 378 
initial, 372, 378 
Assignment, stable, 343 
optimal, 343 
Assignment of jobs, 565 
Assignments, frequency, 732 
Assignment statements, A-1 1—A-12 
Associated homogenous recurrence relations, 520 
Associative laws, 27 
for addition, A-1 
for Boolean algebra, 815, 818 
for lattices, 637 
for multiplication, A-1 
for propositions, 27 
for sets, 129 
Associativity 
for integers modulo m, 243 
Asymmetric relation, 582 
Asymptotic functions, 218 
Asynchronous transfer mode (ATM), 149 
AT&T network, 682 
Autokey cipher 
keystream, 309 


Autokey ciphers, 309 
Automated theorem proving, 114 
Automaton 
deterministic, 872, 873 
deterministic finite-state, 899 
finite-state, 866-872 
nondeterministic, 873 
linear bounded, 886 
nondeterministic, 873 
nondeterministic finite-state, 899 
pushdown, 886 
quotient, 878 
set not recognized by, 885 
Average-case complexity of algorithms, 482-484 
Average-case time complexity, 220, 232 
of linear search, 221 
Avian influenza, 476 
AVL-trees, 808 
Award 
Turing, 854 
Axiom(s), 185, A-1-A-5 
Completeness, A-2 
field, A-1 
for real numbers, A-1—A-2 
for set of positive integers, A-5 
in proving basic facts, A-3—A-5 
of Mathematical Induction, A-5 
order, A-2 
Zermelo—Fraenkel, 176 


B-tree of degree k, 805 
Bézout coefficients, 270, 306 
Bézout’s identity, 270 
Bézout’s theorem, 269, 306 
Bézout, Etienne, 269 
Bachmann, Paul, 206 
Bachmann, Paul Gustav Heinrich, 207 
Back edges, 788 
Back substitution 
solving systems of linear congruences using, 279 
Backtracking, 787-789, 804 
applications of, 791-793 
in directed graphs, 793 
Backus, John, 854 
Backus—Naur form, 899 
Backus—Naur form (BNF), 854 
Backward differences, 513 
Backward substitution 
iteration using, 160 
Bacon, Kevin, 680-681 
Bacon number, 680 
Bacteria, 168, 501 
Balanced rooted trees, 753, 804 
Balanced strings of parentheses, 382 
Balanced ternary expansions, 256 
Bandwidth of graph, 741 
Bar 
chocolate, 342 
Barber paradox, 16 
Base b 
expansions, 246, 306 
algorithm for constructing, 248 
Miller’s test for, 286 
pseudoprime to the, 282 
strong pseudoprime to, 286 
Base-10 expansions, 246 
Base conversion, 247 
Bases of DNA molecules, 389 
BASIC, 391 
Basis, vertex, 691 
Basis step, 313, 333, 349, 377, 752, 767 
Basketball, 434 
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Bayes’ theorem, 468-475 
generalized, 472 
Bayes, Thomas, 470, 472 
Bayesian spam filters, 472-475 
Begging the question, 90, 110 
Bell, E. T., 618 
Bellman, Richard, 508 
Bell numbers, 618 
Bernoulli’s inequality, 330 
Bernoulli, James, 458, 459 
Bernoulli family tree, 745 
Bernoulli trial, 479, 494 
and binomial distribution, 458-460 
Bernstein, Felix, 175 
Bi-implication, 9 
Biconditional, 9-10, 110 
implicit use of, 10 
statement, 9 
truth tables for, 9 
Biconnected graph, 684 
Bicycle racers, 475 
Bidirectional bubble sort, 233 
Bienaymé’s formula, 489 
Bienaymé, Irenée-Jules, 489 
big-O estimates 
for combinations of functions, 212 
for exponential functions, 212 
of important functions, 209 
of n!, 210 
for polynomials, 209,212 
for products of functions, 213 
for sums of functions, 212 
of logarithms, 212 
Big-O notation, 205-232 
careless use of, 216 
origin of, 206 
Big-O relationship 
witnesses to, 205 
big-Omega (Q) notation, 214, 232 
big-Theta (©) notation, 215, 232 
Big-Theta estimates for polynomials, 216 
Bigit, 11 
Bijection, 144, 186 
Bijective proofs, 412 
Binary coded decimal expansion, 836 
Binary digit, 11 
Binary expansions, 246 
Binary insertion sort, 196, 203 
Binary relation, 573, 633 
Binary representation, 306 
Binary search 
worst-case complexity, 220 
Binary search algorithm, 194, 232, 528 
recursive, 363 
Binary search trees, 757-759, 804 
Binary trees, 352-353, 748, 749 
definition of, 804 
extended, 352 
full, 352-353 
height of, 355 
number of vertices of, 355 
with prefix code, 762-763 
Binit, 11 
Binomial coefficients, 410, 415-421, 439 
extended, 539 
Binomial distribution, 458-460 
Binomial expression, 415 
Binomial theorem, 415-418, 439 
extended, 540 
Binomial trees, 805 
Bipartite graphs, 656-658, 736 
coloring of, 657 
Bird flu, 476 
Birthday problem, 461-463 
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Bit, 11, 110 
origin of term, 11 
parity check, 290 
Bit operation(s), 11,110 
Bits, 149 
Bit string, 12, 110 subitem representing set with, 134 
Bit strings 
counting, 388 
without consecutive Os, 505 
decoding, 762-763 
generating next largest, 437 
in interstellar communication, 476 
length of, 505 
Bitwise operation, 110 
Bitwise operators, 12 
Block 
Sudoku, 32 
Block cipher, 297, 306 
Blocks of statements, A-14—A-15 
BNF (Backus—Naur form), 854 
Boat, 692 
Bob, 302, 305 
Bonferroni’s inequality, 467 
Book number of graph, 744 
Boole’s inequality, 467 
Boole, George, 3, 5, 472, 811 
Boolean algebra, 811-846 
abstract definition of, 815 
definition of, 843 
identities, 27 
identities of, 814-816 
Boolean expressions, 812-814, 843 
dual of, 816, 843 
Boolean functions, 811-819, 843 
dual of, 816, 843 
functionally complete set of operators for, 821 
implicant of, 832 
minimization of, 828-840, 843 
representing, 813-821 
self—dual, 844 
threshold, 845 
Boolean power of a square zero—one matrix, 183 
Boolean product, 811-812, 821, 843 
of two zero—one matrices, 182 
Boolean product of zero—one matrices 
algorithm for, 223 
Boolean searches, 18 
Boolean sum, 811-812, 821, 843 
Boolean variable, 11, 110, 814, 820, 843 
Boruvka, Otakar, 800 
Bots, 794 
Bottom-up parsing, 853 
Bound 
greatest lower, 634 
least upper, 634, A-2 
poset, 625 
of lattice, 637 
of poset, 625-626, 634 
of poset, 625-626, 634 
upper, A-2 
Bounded lattices, 637 
Bound occurrence of variable, 44 
Bound variable, 44, 110 
Boxes, distributing objects into, 428-431 
distinguishable, 428 
indistinguishable, 428 
Breadth-first search, 789-791, 804 
Breaking codes, 296 
Bridge, in graphs, 683 
Bridge problem, K6nigsberg, 693-694, 696, 697 
Brute-force algorithm, 224 
Brute-force algorithm for closest pair of points, 225 
Brute-force paradigm, 232 
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Bubble sort, 196, 232 
bidirectional, 233 
worst-case complexity, 221 

Bug, computer, 872 


Building arguments using rules of inference, 73 


Buoyancy, principle of, A-4 
Busy beaver function, 896, 899 
Butane, 750 

Byte, 149 


Cabbage, 692 
Cactus, 806 
CAD programs, 834 
Caesar, Julius, 294 
Caesar cipher, 294 
decryption, 295 
encryption, 294 
key, 295 
Cake, division of, 331 
Calculus 
predicate, 40 
propositional, 3 
Call graphs, 646 
connected components of, 682 
Canterbury Puzzles, The (Dudeney), 504 
Cantor’s theorem, 177 
Cantor, Georg, 117, 118, 173, 176 
Cantor diagonalization argument, 173, 186 
Cantor digits, 438 
Cantor expansion, 256, 438 
Cardano, Girolamo, 445 
Card hands, 411, 429 
Cardinality, 121, 170-175, 185, 186 
of a countable set (Ng), 171, 186 
of union of sets, 128 
Cardinality of the set of real numbers, 186 
Cardinal number, 121 
Carmichael, Robert Daniel, 283 
Carmichael number, 283, 306 
Carroll, Lewis (Charles Dodgson), 50 
Carry, 250 
Cartesian product, 123 
Cartesian product of more than two sets, 123 
Cases, proof by, 92-110 
Cash box, 424-425 
Catalan, Eugéne Charles, 507 
Catalan numbers, 507 
Caterpillar, 807 
Cathy, 305 
Cayley, Arthur, 745, 750 
Ceiling function, 149, 186 
properties of, 150 
Celebrity problem, 332 
Cells, adjacent, 830 
Center, 757 
Ceres, 241 
Chain, 637 
Chain letter, 753 
Character cipher, 297, 306 
Characteristic equation, 515 
Characteristic roots, 515 
Chebyshev’s inequality, 491, 495 
Chebyshev, Pafnuty Lvovich, 490, 491 
Checkerboard 
tiles, 103 
Check digit, 289-292 
for airline ticket identification numbers, 293 
ISBN, 291 
ISBN-13, 308 
ISSN, 293 
RTN, 308 
UPC, 290 
Checkerboard, 103 


Checkerboard, tiling of, 326, 333 
Cheese, in Reve’s puzzle, 504 
Chen, J. R., 264 
Chess, game tree for, 769 
Chessboard 
n-queens problem on, 792-793 
knight’s tour on, 707 
squares controlled by queen on, 740 
Chi (Greek letter), 728 
Chickens, 741 
Child of vertex, 747, 803 
Chinese meal, preparing, 635 
Chinese postman problem, 698, 744 
Chinese remainder theorem, 277, 306 
Chocolate bar, 342 
Chomp, 98, 342, 638 
winning strategy for, 98 
Chomsky, Avram Noam, 847, 851, 853 
Chomsky classification of grammars, 851-852 
Chromatic number, 728, 737 
k-critical, 734 
edge, 734 
of graph, 728 
Chung-Wu Ho, 340 
Church, Alonzo, 897 
Church-Turing thesis, 893 
Cipher 
affine, 296, 306 
autokey, 309 
block, 306 
Caesar, 294 
character, 297, 306 
monoalphabetic, 297 
shift, 295, 306 
transposition, 297 
Vigenére, 304 
Cipher block, 297 
Circuits 
combinational, 823 
depth of, 828 
examples of, 823-825 
minimization of, 828-840 
digital, 21 
Euler, 693-698, 736 
Hamilton, 698-703, 736 
in directed graph, 599, 633 
in graph, 679 
logic, 20, 110 
multiple output, 826 
simple, 679 
Circular permutation, 415 
Circular reasoning, 90, 110 
Circular relation, 636 
Circular table, 394 
Citation graphs, 646 
Civil War, 38 
C language identifier, 857 
Class 
congruence, 241 
equivalence, 610-611, 633 
and partitions, 612-614 
definition of, 610 
representative of, 610 
of nondeterministic polynomial-time problems, 
900 
NP, 227, 896 
of NP-complete problems, 897, 900 
of polynomial-time problems, 227, 896, 900 
Class P, 227, 896, 900 
Class A addresses, 392 
Class B addresses, 392 
Class C addresses, 392 
Class D addresses, 392 
Class E addresses, 392 
Clause, 74 


Climbing rock, 163 
Clique, 739 
Clock, 240 
Closed formula 
for terms of a sequence, 159 
Closed interval, 117 
Closed walk, 679 
Closest-pair problem, 532-535 
Closure 
for integers modulo m, 243 
Kleene, 899 
closure 
Kleene(, 866 
Closure, Kleene, 866 
Closure laws 
for addition, A-1 
for multiplication, A-1 
Closures of relations, 597-606, 633 
teflexive, 598, 633 
symmetric, 598, 634 
transitive, 597, 600-603, 634 
computing, 603-606 
Coast Survey, U. S., 38 
COBOL, 854, 872 
Cocks, Clifford, 299 
Code 
breaking, 296 
Codes, Gray, 702-703 
Codes, prefix, 762-764, 804 
Codeword enumeration, 506 
Coding, Huffman, 763-764, 804 
Codomain 
of a function, 139, 186 
of partial function, 152 
Coefficient(s) 
Bézout, 270, 306 
binomial, 409, 415-421, 439 
constant 
linear homogenous recurrent relations with, 
515-520, 565 
linear nonhomogenous recurrent relations with, 
520-524, 565 
extended, 539 
multinomial, 434 
Collaboration graphs, 645 
paths in, 680 
Collatz problem, 107 
Collection of sets 
intersection of, 133 
union of, 133 
Collision 
in hashing, 288 
in hashing functions, probability of, 462-463 
Coloring 
chromatic number in, 728 
of bipartite graphs, 657 
of graphs, 727-732, 736 
of maps, 727 
Combinational circuits, 823 
depth of, 828 
examples of, 823-825 
minimization of, 828-840 
Combinations, 409-413 
generating, 437-438 
of events, 449-450, 455-456 
with repetition, 424-427 
Combinatorial identities, 417-421 
Combinatorial proof, 412, 439 
Combinatorics, 385, 404, 431, 439 
Comments in pseudocode, A-12 
Common difference, 157 


Common errors 
with exhaustive proofs, 95 
with proofs by cases, 95 
Common ratio, 157 
Commutative group, 244 
Commutative laws, 27 
for addition, A-1 
for Boolean algebra, 815, 818 
for lattices, 637 
for multiplication, A-1 
for sets, 129 
Commutative ring, 244 
Commutativity 
for integers modulo m, 243 
Comparable elements in poset, 619, 633 
Compatibility laws, A-2 
Compatible total ordering, 628, 634 
Compilers, 611, 872 
Complement, 811, 843 
double, law of, in Boolean algebra, 815, 818, 819 
of Boolean function, 811 
Complementary event, 449 
Complementary graph, 667 
Complementary relation, 582 
Complementation law, 129 
Complemented lattice, 637, 817 
Complement law, 129 
Complement of a fuzzy set, 138 
Complement of a set, 128, 129, 186 
Complete m-ary tree, 756 
Complete m-partite graph, 738 
Complete bipartite graph, 658, 736 
Complete graphs, 655, 736 
Complete induction, 334 
Complexity 
computational, 219 
constant, 225 
exponential, 225 
factorial, 225 
linear, 225 
linearithmic, 225 
logarithmic, 225 
of algorithm for Boolean product of zero—one 
matrices, 223 
of matrix multiplication, 223 
polynomial, 225 
space, 219, 232 
time, 219, 232 
Complexity of algorithms, 218-229 
average-case, 482-484 
computational 
of Dijkstra’s algorithm, 714 
of breadth-first search algorithms, 791 
of depth-first search algorithms, 789 
of merge sort, 532 
Complexity of merge sort, 532 
Complex numbers 
set of, 116 
Components of graphs, connected, 682-685 
strongly, 686, 736 
Composite, 257 
Composite integer, 306 
Composite key, 585, 633 
Composite of relations, 580, 633 
Composition of functions, 146, 186 
Composition rule, 373 
Compound interest, 160 
Compound proposition, 25, 109 
dual of, 35 
satisfiable, 30, 110 
Compound propositions, 3 
consistent, 110 
equivalences of, in Boolean algebra, 812 
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equivalent, 8 

logically equivalent, 25, 110 

truth tables of, 10 

well-formed formula for, 354 
Computable 

function, 175 
Computable function, 177, 186, 896, 900 
Computable numbers, 886 
Computation, models of, 847-897 
Computational complexity, 219 
Computational complexity of algorithms 

of Dijkstra’s algorithm, 714 

Turing machine in making precise, 894 
Computational geometry, 338-340, 532-535 
Computer-aided design (CAD) programs, 834 
Computer arithmetic with large integers, 279 
Computer debugging, 872 
Computer file systems, 750 
Computer network, 641 

interconnection networks, 661-663 

local area networks, 661 

multicasting over, 786 

with diagnostic lines, 642 

with multiple lines, 642 

with multiple one-way lines, 642 

with one-way lines, 643 
Computer programming, 854 
Computer representation of sets, 134 
Computer science, 854 
Computer time used by algorithms, 228 
Computer virus 

invention of term, 300 

transmission, 441 
Concatenation, 350, 866, 899 
Conclusion, 70, 110 
Conclusion of a condition statement, 6 
Concurrent processing, 647 
Condition 

necessary, 6 

sufficient, 6 
Conditional constructions, A-12—A-13 
Conditional probability, 453, 456-457, 494 
Conditional statement(s), 6-9 

contrapositive, 110 

contrapositive of, 8 

converse, 110 

converse of, 8 

for program correctness, 373-375 

inverse of, 8 

truth table for, 6 
Conditions 

don’t care, 836-837 

initial, 158, 565 
Congruence, 240, 306 

applications of, 287-292 

linear, 275, 306 
Congruence class, 241 
Congruence modulo m, 609-610, 633 
Congruence quadratic, 285 
Congruent to, 240, 306 
Conjecture, 81, 110 

3x + 1, 107 

Frame’s, 513 

Goldbach’s, 264 

twin prime, 264 
Conjectures about primes, 263 
Conjunction, 4, 110, 843 

distributive law of disjunction over, 27 

negating, 28 

truth table for, 4 
Conjunction (rule of inference), 71 
Conjunctive normal form, 820 
Connected components of graphs, 682-685 

strongly, 686, 736 
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Connected graphs, 678-689 
directed, 685-687 
planar simple, 719-723 
strongly, 685 
undirected, 681-685 
weakly, 686 
Connecting vertices, 651 
Connectives 
logical, 4 
Connectivity 
edge, 684 
vertex, 684 
Connectivity relation, 600, 633 
Consequence of a condition statement, 6 
Consistent compound propositions, 110 
Consistent system specifications, 18 
Constant coefficients 
linear homogenous recurrent relations with, 
515-520, 565 
linear nonhomogenous recurrent relations with, 
520-524, 565 
Constant complexity, 225 
Constructing logical equivalences, 29 
Construction 
automaton that recognizes a regular set, 881 
Construction of the real numbers, A-5 
Constructions 
conditional, A-12—A-13 
loop, A-14—A-15 
Constructive existence proof, 96, 110 
Contains, 116 
Context-free grammar(s), 851, 886 
ambiguous, 901 
Context-free language, 851 
Context-sensitive grammars, 851 
Context-sensitive languages, 886 
Contingency, 25, 110 
Continuum hypothesis, 175, 186 
Contraction, edge, 664 
Contradiction, 25, 110 
proof by, 110 
Contradiction, proof by, 86 
Contraposition, 83 
proof by, 110 
Contrapositive, 8, 110 
of a conditional statement, 110 
Control unit 
Turing machine, 889 
Converse, 8, 110 
of a conditional statement, 110 
of directed graph, 668 
Conversion 
between bases, 247 
between binary and hexadecimal, 249 
between binary and octal, 249 
Convex polygon, 338 
Cook, Stephen, 227 
Cook—Levin theorem, 227 
Cookie, 426 
Corollary, 110 
Correctness 
of an algorithm, 193 
of programs, 372-378 
conditional statements for, 373-375 
loop invariants for, 375-376 
partial, 372 
program verification for, 372-373 
rules of inference for, 373 
of recursive algorithms, 364-365 
Correspondence 
one-to-one, 144, 186 
Countability of set of rational numbers, 172 


Countable set, 171, 186 
cardinality of, 186 
Counterexample(s), 88, 102, 110 
Counting, 385-444, 501-571 
basic principles of, 385-390 
bit strings, 388 
without consecutive Os, 505 
Boolean functions, 814 
by distributing objects into boxes, 428-431 
combinations, 409-413 
derangements, 562, 566 
functions, 387 
generating functions for, 541-546 
Internet addresses, 392 
one-to-one functions, 387 
onto functions, 560-562, 566 
passwords, 391 
paths between vertices, 688-689 
permutations, 407-409 
pigeonhole principle and, 399-405 
reflexive relations, 578 
relations, 578-579 
subsets of finite set, 388 
telephone numbers, 387 
tree diagrams for, 394-395 
variable names, 391 
with repetition, 423 
Covariance, 494 
Covering relation, 623, 631 
Covers, 631 
CPM (Critical Path Method), 639 
C programming language, 398, 611 
Crawlers, 794 
Cricket, 97 
Criterion 
Euler’s, 286 
Critical Path Method (CPM), 639 
Crossing number, 726 
Cryptanalysis, 296, 306 
for shift cipher, 296 
Vigenére cipher, 305 
Cryptographic protocols, 302 
Cryptosystem, 297-306 
definition of, 297 
private key, 298, 306 
public key, 298 
RSA, 299, 306 
shift cipher, 298 
Cunningham numbers, 262 


Cut 
edge, 683, 684 
set, 806 
vertex, 683, 684 
Cycle 


in directed graph, 599, 633 
with n vertices, 655 
Cylinder, 831 
Czekanowski, Jan, 800 


Database 

composite key of, 585 

intension of, 585 

primary key of, 585 

records in, 584 

relational model of, 584-586, 633 
Database query language, 588-589 
Data compression, 763 
Datagrams, 399 
Datalogy, 854 
Datatype, 117 
David Hilbert, 171 
de Bruijn sequences, 744 
Debugging, 872 


Decidable problem, 895 
Decimal expansion of a real number, 174 
Decimal expansions, 246 
Decision problems, 894, 900 
Decision trees, 760-762, 804 
Deck, standard, 402 
Decreasing function, 143 
Decryption, 295, 306 
affine cipher, 296 
Caesar cipher, 295 
RSA, 300 
Decryption key 
RSA system, 301 
Dedekind, Richard, 175 
Deductive reasoning, 312 
Deep-Blue, 769 
Deferred acceptance algorithm, 204, 343 
Definiteness 
of an algorithm, 193 
Definition 
recursive, 311, 344-357 
Degree 
of n-ary relations, 584 
of linear homogenous recurrence relations, 514 
of region, 722 
of vertex in undirected graph, 652 


Degree-constrained spanning tree, 806 
Degree of membership in a fuzzy set, 138 


de la Vallée-Poussin, Charles-Jean-Gustave-Nicholas, 


262 

Delay machine, 861-862 
de Méré, Chevalier, 452 
De Morgan’s laws, 26, 27 
or Boolean algebra, 815, 819 
or propositions, 31, 28-31 
‘or quantifiers, 47 
or sets, 129 
proving by mathematical induction, 323-324 
De Morgan, Augustus, 26, 29, 729 
Dense 
graph, 670 
poset, 632 
Denying the hypothesis, fallacy of, 75 
Dependency notation, 846 
Depth-first search, 787-789, 804 

applications of, 794-795 

in directed graphs, 794-795 
Depth of combinatorial circuit, 828 
Derangements, 562-565 

number of, 562, 566 
Derivable, 899 

directly, 899 
Derivable from, 849 
Derivation, 849 
Derivation tree, 899 
Derivation trees, 852-854 
Descartes René, 122 
Descendants of vertex, 747, 804 
Describing a set 

by listing its members, 116 

by the roster method, 116 

using set builder notation, 116 


Designing finite-state automata, 869 
Detachment 

laws of, 71 
Deterministic finite-state automata, 872, 899 
Deviation, standard, 487 
Devil’s pair, 678 
Diagnostic test results, 471-472 
Diagonalization argument, Cantor, 173, 186 
Diagonal of a polygon, 338 
Diagonal of a square matrix, 181 
Diagonal relation, 598 


Diagrams 
Hasse, 622-626, 634 
state, for finite-state machine, 860 
tree, 394-395, 439 
Venn, 118, 185 
Diameter of graph, 741 
Dice, 446 
Dictionary ordering, 435 
Die, 468, 488-489 
dodecahedral, 496 
octahedral, 496 
Difference, A-6 
backward, 513 
common, 157 
orward, 568 
Difference equation, 513 
Difference of multisets, 138 
Difference of sets, 128, 186 
Diffie, Whitfield, 302 


Diffie-Helman key agreement protocol, 302 


Digit 
binary, 11 
Digital circuit(s), 20-21 
Digitial signatures, 303, 306 
Digitial signatures in RSA system, 303 
Digits 
Cantor, 438 
Digraphs, 633, 643, 735 
circuit (cycle) in, 599, 633 
connectedness in, 685-687 
converse of, 668 
depth-first search in, 794-795 
edges of, 643, 653-654 
Euler circuit of, 694 
loops in, 594, 633 
paths in, 599-600, 633, 736 
representing relations using, 594-596 
self—converse, 740 
vertex of, 641, 654 
Dijkstra’s algorithm, 709-714, 737 
Dijkstra, Edsger Wybe, 710 
Dimes, 199 
Diophantus, 106 
Dirac’s theorem, 701 
Dirac, G. A., 701 
Directed edges, 644, 653-654, 735 
Directed graphs, 633, 643, 735 
circuit (cycle) in, 599, 633 
connectedness in, 685-687 
converse of, 668 
depth-first search in, 794-795 
edges of, 643, 653-654 
Euler circuit of, 694 
loops in, 594, 633 
paths in, 599-600, 633, 736 
representing relations using, 594-596 
self—converse, 740 
vertex of, 641, 654 
Directed multigraph, 643, 735 
paths in, 679 
Directly derivable, 849, 899 
Direct proof, 82, 110 
Dirichlet, G. Lejeune, 262, 400 
Dirichlet drawer principle, 400 
Discrete logarithm, 284, 306 
Discrete logarithm problem, 284 
Discrete mathematics 
definition of, vii, viii, xviii 
problems studied in, xviii 
reasons to study, xviii 
Discrete probability, 445-500 
assigning, 453-455 
conditional, 453, 456-457, 494 


finite, 445-448 
Laplace’s definition of, 445 
of collision in hashing functions, 462-463 
of combinations of events, 449-450, 455-456 
Disjoint sets, 128 
Disjunction(s), 4, 109, 843 
associative law of, 27 
distributive law of, over conjunction, 27 
negating, 28 
truth table for, 4 
Disjunctive normal form, 35 
for Boolean variables, 820 
Disjunctive syllogism, 71 
Disquisitiones Arithmeticae (Gauss), 241 
Distance 
between distinct vertices, 741 
between spanning trees, 797 
Distinguishable 
boxes, 428 
objects, 428 
strings, 888 
Distributing objects into boxes, 428-431 
Distribution 
binomial, 458—460 
of random variable, 460, 494 
geometric, 484-485, 494 
probability, 453 
uniform, 454, 494 
Distributive lattice, 637, 817 
Distributive laws, 27 
for Boolean algebra, 815, 818 
for propositions, 27 
for sets, 129 
Distributivity 
for integers modulo m, 243 
Divide-and-conquer 
algorithms, 527, 565 
recurrence relations, 527-535 
Dividend, 238, 239 
Divides, 238, 306 
Divine Benevolence (Bayes), 472 
Divisibility facts, proving, by mathematical 
induction, 321 
Divisibility relation, 619 
Division 
of integers, 238-240 
trial, 258 
Division algorithm, 239, 306 
Division of a cake, 331 
Division rule, 394 
Division rule for counting, 394 
Divisor, 238, 239 
greatest common, 265-267, 306 
DNA (deoxyribonucleic acid), 388 
DNA sequencing, 698 
Dodecahedral die, 496 
Dodgson, Charles Lutwidge (Lewis Carroll), 50 
Domain 
of n-ary relation, 584 
of a function, 139, 186 
of a quantifier, 40 
of partial function, 152 
of relation, 584 
restricted 
of quantifier, 44 
Domain of definition 
of partial function, 152 
Domain of discourse, 40, 110 
Dominating set, 739 
Domination laws, 27 
for Boolean algebra, 815 
for sets, 129 
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Domino(s), 103, 314 
Don’t care conditions, 836-837 
Double complement, law of, in Boolean algebra, 815, 


819 


Double counting proofs, 412 
Double hashing, 292 
Double negation law, 27 
Drug testing, 475 

Dual 


of a compound proposition, 35 
of Boolean expression, 816 

of Boolean function, 816 

of poset, 630 


Dual graph, 727 

Duality in lattice, 639 

Duality principle, for Boolean identities, 816 
Dudeney, Henry, 504 

Dynamic programming, 507 


Ear(s), 343 


nonoverlapping, 343 


Earth, 476 

EBNF (extended Backus—Naur form), 858 
Eccentricity of vertex, 757 

Ecology, niche overlap graph in, 648 


Ed 


Ed 
Ed 
Ed 
Ed 
Ed 


ge 
adding from a graph, 664 
removing a vertices from, 664 
removing from a graph, 663 
ge chromatic number, 734 

ge coloring, 734 

ge connectivity, 684 

ge contraction, 664 

ge(s) 

cut, 683, 684 

directed, 644, 654, 735 
endpoints of, 651 

incident, 651, 735 

multiple, 642, 643, 735 

of directed graph, 643, 653-654 
of directed multigraph, 643 

of multigraph, 642 

of pseudograph, 643 

of simple graph, 642, 667 

of undirected graph, 644, 654 
undirected, 644, 735 


Edge vertex, 594 
Effectiveness 


of an algorithm, 193 


Egyptian (unit) fraction, 380 
Einstein, Albert, 24 
Electronic mail, 472 
Element 


image of, 186 
pre-image of, 186 


Elementary subdivision, 724, 736 
Element of a set, 116, 185 
Elements 


comparable, in partially ordered set, 619, 633 
equivalent, 608 

fixed, 494 

greatest, of partially ordered set, 625, 634 
incomparable, in partially ordered set, 619, 633 
least, of partially ordered set, 625, 634 
maximal, of partially ordered set, 624, 634 
minimal, of partially ordered set, 624, 625, 634 


Elements of Mathematical Logic (Lukasiewicz), 782 
Ellipses (...), 116 

Empty folder, 118 

Empty set, 118, 185 

Empty string(s), 157, 186, 849 
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Encryption, 294, 306 
affine transformation, 296 
public key, 306 
RSA, 299 
Encryption key, 306 
Encrytion 
Caesar cipher, 294 
Endpoints of edge, 651 
Engine 
Analytic, 31 
Entry of a matrix, 178 
Enumeration, 392, 439 
codeword, 506 
Equal functions, 139 
Equality 
of sets, 117, 185 
Equal matrices, 178 
Equation 
characteristic, 515 
difference, 513 
Equivalence 
proof of, 87 
Equivalence(s) 
logical, 27 
Equivalence classes, 610-611, 633 
and partitions, 612-614 
definition of, 610 
representative of, 610 
Equivalence relations, 607-614, 633 
definition of, 608 
Equivalent 
compound propositions, 8 
logically 
compound propositions, 25 
Equivalent Boolean expressions, 813 
Equivalent elements, 608 
Equivalent finite-state automata, 868, 871-872 
Eratosthenes, 259, 560 
sieve of, 259, 306, 565 
Erdés, Paul, 260, 263, 635, 636, 680 
Erd6s number, 635, 680, 689 
Erdés number Project, 680 
Error 
single, 291 
transposition, 291 
Errors, 
in exhaustive proofs, 95 
in proofs by cases, 95 
in proofs by mathematical induction 328-329 
Essential prime implicant, 832, 843 
Euclid, 267 
Euclidean algorithm, 267, 347 
Euler ¢-function, 272 
Euler’s criterion, 286 
Euler’s formula, 720-723, 737 
Euler’s formula‘Eureka,”, A-4 
Euler, Leonhard, 693, 695 
Euler circuits, 693-698, 736 
Euler paths, 693-698, 736 
Evaluation functions, 769 
Even, 83 
Event(s), 446 
combinations of, 449-450, 455-456 
complementary, 449 
independent, 452, 457-458, 494 
mutually independent, 497 
Exams, scheduling, 731, 732 
Exchange 
key, 302 
Exclusion rule, 349 
Exclusive or, 5, 6, 110 
truth table for, 6 


Exercises 

difficult, xix 

extremely challenging, xix 

result used in book, xix 

routine, xix 
Exhaustive proof, 93, 110 

common errors with, 95 
Existence proof(s), 96 

constructive, 96, 110 

nonconstructive, 96, 110 
Existential generalization, 76 
Existential instantiation, 76 
Existential quantification, 42, 110 
Existential quantifier, 42 
Expansion(s) 

balanced ternary, 256 

base-b, 246 

binary, 246 

binary coded decimal, 836 

Cantor, 256 

decimal, 246 

hexadecimal, 246 

octal, 246 
Expected values, 477-480, 494 
in hatcheck problem, 481 
linearity of, 477-484, 494 
of inversions in permutation, 482-484 
Experiment, 446 
Exponential complexity, 225 
Exponential functions, A-7 
big-O estimates for, 212 
Exponential generating function, 551 
Exponentiation 
modular, 253 
recursive, 363 
Expression(s) 

binomial, 415 

Boolean, 812-814, 843 

infix form of, 780 

postfix form of, 781 

prefix form of, 780 

regular, 879, 899 
Expressions 

logically equivalent, 110 
extended Backus—Naur form, 858 
Extended binary trees, 352 
Extended binomial coefficients, 539 
Extended binomial theorem, 540 
Extended Euclidean algorithm, 270, 273 
Extended transition function, 867 
Extension 

of transition function, 867 
Exterior of simple polygon, 338 


Facebook, 645 

Factor, 238 

Factorial complexity, 225 
Factorial function, 151 
Factorials, recursive procedure for, 361 
Factoring, 262 

Factorization 

prime, 259 

Failure, 458 

Fairy, tooth, 112 

Fallacies, 69, 75, 110 

of affirming the conclusion, 75 
of denying the hypothesis, 75 
False 

negative, 471, 472 

positive, 471, 472 

Family trees, 745 

Farmer, 692 


Fast multiplication 
of integers, 528-529 
of matrices, 529 
Female pessimal, 234 
Fermat’s last theorem, 106, 349 
Fermat’s little theorem, 281, 306 
proof of, 285 
Fermat, Pierre de, 106, 281, 282 
Fibonacci, 348 
Fibonacci numbers, 347 
and Huffman coding, 771 
formula for, 517 
iterative algorithm for, 366 
rabbits and, 502-503 
recursive algorithms for, 365 
Fibonacci sequence, 158 
Fibonacci trees, rooted, 757 
Field axioms, A-1 
Fields, 584 
Fields Medal, 263 
Filter, spam, 472-475 
Final assertion, 372, 378 
Final exams, scheduling, 731, 732 
Final state 
finite-state automaton, 867 
of a Turing machine, 891 
Turing machine, 891 
Final value, A-14 
Finding maximum element in a sequence 
algorithm for, 192 
Finite-state automata, 866-872 
accepting state, 867 
designing, 869 
deterministic, 872, 899 
equivalent, 868 
final state, 867 
initial state, 867 
minimization, 872 
nondeterministic, 873, 899 
set not recognized by, 885 
Finite-state machine, 847, 858, 885, 899 
for addition, 862 
input string, 861 
output string, 861 
transition function extension, 867 
transition function extension in, 867 
with output, 859, 865 
with outputs, 859, 863 
Finite-state transducer, 859 
Finite graph, 641 
Finiteness 
of an algorithm, 193 
Finite probability, 445-448 
Finite set, 121, 185 
Finite sets, 553, 565 
number of subsets, 323 
subsets of 
counting, 388 
number of, 323 
union of three, number of elements in, 554-556, 
566 
union of two, number of elements in, 553, 565 
First difference, 513 
First forward difference, 568 
Fixed elements, 494 


Fixture controlled by three switches, circuit for, 825 


Flavius Josephus, 512 

Fleury’s algorithm, 697, 706 

Flip-flops, 846 

Floor function, 149, 186 
properties of, 150 

Floyd’s algorithm, 717 


Folder 
empty, 118 
Forbidden pairs, 234 
Forests, 746 
definition of, 803 
minimum spanning, 802 
spanning, 796 
Form 
argument, 70 
Backus-Naur, 854, 899 
conjunctive normal, 820 
disjunctive normal 
for Boolean variables, 820 
extended Backus—Naur form, 858 
infix, 780 
postfix, 781 
prefix, 780 
Form, argument, 110 
Formal language, 848 
Formal power series, 538 
Formula(s) 
Euler’s, 720-723, 737 
for compound propositions, 354 
for Fibonacci numbers, 517 
of operators and operands, 351 
Stirling’s, 151 
summation, 315 
well-formed, 350 
FORTRAN, 854 
Fortune cookie, 441 
Forward differences, 568 
Forward reasoning, 100 
Forward substitution 
iteration using, 160 
Four color theorem, 728-731, 737 
Fraction, unit (Egyptian), 380 
Frame’s conjecture, 513 
Free variable, 44, 110 
Frend, Sophia, 29 
Frequency assignments, 732 
Friendship graph, 645 
Frisbee, rocket-powered, 812 
Full m-ary tree, 748, 752, 804 
complete, 756 
Full adder, 826, 843 
Full binary trees, 352-353 
height of, 355 
number of vertices of, 355 
Full subtractor, 828 
Function(s), 139, 186 
Addition of, 141 
Ackermann’s, 359 
asymptotic, 218 
as relations, 574 
big-O estimates of, 209 
bijective, 144 
Boolean, 811-819, 843 
dual of, 816 
functionally complete set of operators for, 821 
implicant of, 832 
minimization of, 828-840, 843 
representing, 813-821 
self—dual, 844 
threshold, 845 
busy beaver, 896,899 
ceiling, 149, 186 
codomain of, 139 
codomain of a, 186 
composition of, 146, 186 
computable, 175, 177, 186, 896, 900 
counting, 387 
decreasing, 143 
domain of, 186 


equal, 139 
Euler ¢, 272 
evaluation, 769 
exponential, A-7 
factorial, 151 
floor, 149, 186 
generating, 537-548, 565 
exponential, 551 
for counting, 541-546 
for proving identities, 548 
for recurrence relations, 546-548 
probability, 552 
graph of, 148 
greatest integer, 149 
growth of, 209 
growth of combinations of, 212-214 
hashing, 287 
collision in, probability of, 462-463 
identity, 144 
increasing, 143 
injective, 141 
counting, 387 
integer-valued, 140 
inverse, 145 
inverse of, 186 
invertible, 145 
iterated, 360 
iterated logarithm, 360 
logarithmic, A-7—A-9 
McCarthy, 380 
multiplication of functions, 141 
number-theoretic, 892 


mod, 239, 306 
one-to-one, 141, 186 
onto, 186 


number of, 560-562, 566 

partial, 152, 186, 889 

probing, 288 

propositional, 110 

range of, 186 

real-valued, 140 

recursive, 345-349, 378 

strictly decreasing, 143 

strictly increasing, 143 

surjective, 143 

threshold, 845 

total, 152 

transition, 859 

Turing machines computing, 892-893 

uncomputable, 175, 186, 896, 900 
Functional completeness, 821 
Functional decomposition, 846 
Functionally complete logical operators, 35 
Functionally complete set of operators 

for Boolean functions, 821, 843 
Fundamental theorem of algebra, 241 
Fundamental theorem of arithmetic, 258, 306 

proof of, 271 
Fuzzy logic, 16 
Fuzzy set(s), 138 

complement of, 138 

degree of membership, 138 

intersection of, 138 

union of, 138 


Gédel, Escher, Bach (Hofstader), 382 
Gale-Shapley algorithm, 204 
Gambling, 445 
Game 

obligato, 112 
Game trees, 764-769, 804 
Gates, logic, 21, 110, 822-827 

AND, 21, 823, 843 

combination of, 823 
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NAND, 828 
NOR, 828 
OR, 21, 823, 843 
threshold, 845 
Gating networks, 822-823 
depth of, 828 
examples of, 823-825 
minimization of, 828-840 
Gauss, Karl Friedrich, 241 
Gecko, Gordon, 198 
Gene, 389 
Generality 
of an algorithm, 193 
Generalization 
existential, 76 
universal, 76 
Generalized combinations, 423-427 
Generalized induction, 356-357 
Generalized permutations, 423 
Generalized pigeonhole principle, 401-403, 439 
Generating functions, 537-548, 565 
exponential, 551 
for counting, 541-546 
for proving identities, 548 
for recurrence relations, 546-548 
probability, 552 
Generator, power, 292 
Gene sequencing, 389 
Genome, 389 
Geometric distribution, 484-485, 494 
Geometric mean, 100, 332 
Geometric progression, 157, 186 
sum of terms of, 164 
Geometric progression(s) 
sums of, 318-319 
Geometric series, 164 
Geometry, computational, 338-340, 532-535 
Giant strongly connected components (GSCC), 686 
GIMPS (Great Internet Mersenne Prime Search, 261 
Givens 
in Sudoku, 32 
Goat, 692 
Goldbach’s conjecture, 264 
Goldbach, Christian, 264 
Golomb’s self-generating sequence, 382 
Golomb, Solomon, 105 
Google, 18, 794 
Gossip problem, by mathematical induction, 332 
Government Communications Headquarters 
(GCHQ), U.K, 299 
Government Communications Headquarters 
(GCHQ), U.K., 299 
Graceful trees, 807 
Graham, Ron, 636 
Grammar(s), 848 
Backus—Naur form of, 854 
context-free, 886 
context-free (type 2), 851 
context-sensitive, 851 
language generated by, 850 
language of, 850 
monotonic, 852 
noncontracting, 852 
productions, 849 
phrase-structure, 849-854, 899 
regular, 884, 899 
regular (type 3), 851 
type 0, 851, 899 
type 1, 851, 899 
type 2, 851, 899 
type 3, 851, 883, 899 
Grand Hotel, Hilbert’s, 171 
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Graph(s), 641-744 simple, 642, 654-655, 735 Hamilton paths, 698-703, 736 
k-connected, 684 coloring of, 727 Handle, 806 
n-regular, 667 connected planar, 719-723 Handshaking theorem, 653 
academic collaboration, 645 crossing number of, 726 Hanoi, tower of, 503-504 
acquaintanceship, 645 dense, 670 Hardware systems, 17 
airline route, 647 edges of, 642, 663 Hardy, Godfrey Harold, 97 
bandwidth of, 741 isomorphic, 671-675, 736 Hardy, Godrey Harold, 97 
biconnected, 684 orientation of, 740 Hardy—Weinberg law, 97 
bipartite, 656-658, 736 paths in, 785 Harmonic mean, 108 
book number of, 744 random, 742 Harmonic number(s) 
call, 646 self-complementary, 677 inequality of, 320-321 
connected components of, 682 sparse, 670 Harmonic series, 321 
chromatically k-critical, 734 thickness of, 726 Hashing 
chromatic number of, 728 vertices of, 642, 663 collision in, 288 
citation, 646 with spanning trees, 785-787 double, 292 
collaboration, 645 simple directed, 643 function, 287 
coloring, 727-732, 736 sparse, 670, 802 key, 287 
complementary, 667 strongly directed connected, 685 Hashing functions 
complete, 655, 736 subgraph of, 663, 736 collision in, probability of, 462-463 
complete m-partite, 738 terminology of, 651-654 Hasse’s algorithm, 107 
complete bipartite, 658, 736 undirected, 644, 653, 736 Hasse, Helmut, 623 
connected components of, 682-685, 736 connectedness in, 681-685 Hasse diagrams, 622-626, 634 
connectedness in, 678-689 Euler circuit of, 694 Hatcheck problem, 481, 562 
cut set of, 806 Euler path of, 698 Hazard-free switching circuits, 846 
definition of, 643 orientation of, 740 Heaps, 809 
diameter of, 741 paths in, 679, 736 Heawood, Percy, 728 
directed, 633, 643, 735 underlying, 654, 736 Height, star, 901 
circuit (cycle) in, 599, 633 union of, 664, 736 Height-balanced trees, 808 
connectedness in, 685-687 very large scale integration, 744 Height of full binary tree, 355 
converse of, 668 Web, 646-647 Height of rooted tree, 753, 804 
dense, 670 strongly connected components of, 686 Hellman, Martin, 302 
depth-first search in, 794-795 weighted, 736 Hexadecimal expansions, 246 
edges of, 643, 653-654 shortest path between, 707-714 Hexadecimal representation, 306 
Euler circuit of, 694 traveling salesman problem with, 714-716 Hexagon identity, 421 
loops in, 594, 633 wheel, 655, 736 Hilbert’s 23 problems, 171, 895 
paths in, 599-600, 633, 736 Graph of a function, 148 Hilbert’s Grand Hotel, 171 
representing relations using, 594-596 Gray, Frank, 703 Hilbert’s Tenth Problem, 895 
self—converse, 740 Gray codes, 702-703 Hilbert, David, 171, 176, 895 
simple, 643 “Greater than or equal” relation, 618-619 HIV, 476 
vertex of, 641, 654 Greatest common divisor, 265-267, 306 Ho, Chung-Wu, 340 
directed multigraph, 644, 735 as linear combination, 269 Hoare, C. Anthony R., 372, 374 
dual, 727 Greatest element of poset, 625, 634 Hoare triple, 372 
finite, 641 Greatest integer function, 149 Hofstader, Douglas, 382 
friendship, 645 Greatest lower bound, 634 Hollywood graph, 645 
Hollywood, 645 of poset, 625 paths in, 680-681 
homeomorphic, 724-725, 736 Great Internet Mersenne Prime Search (GIMPS), 261 Homeomorphic graphs, 724-725, 736 
independence number of, 741 Greedy algorithm(s), 198, 232, 235, 325, 764, 798, Hopper, Grace Brewster Murray, 872 
infinite, 641 804 Hops, 662 
influence, 645 definition of, 198 Horner’s method, 230 
in roadmap modeling, 647 for making change, 199 Horse races, 31 
intersection, 650 for minimizing maximum lateness of a job, 235 Host number (hostid), 392 
isomorphic, 668, 671-675, 736 for scheduling talks, 200 HTML, 858 
paths and, 687-688 Green, Ben, 263 Huffman, David A., 763 
matching in, 659 Green-Tao theorem, 263 Huffman coding, 763-764, 804 
mixed, 644 Group commutative, 244 variations of, 764 
models, 644-649 Growth of functions, 209 Human genome, 389 
module dependency, 647 GSCC (giant strongly connected components), 686 Husbands, jealous, 693 
monotone decreasing property of, 742 Guarding set, 735 Hybrid topology for local area network, 661 
monotone increasing property of, 742 Guare, John, 680 Hydrocarbons, trees modeling, 750 
multigraphs, 642, 644, 735 Guidelines for mathematical induction, 328 Hypercube, 662 
niche overlap, 648 Guthrie, Francis, 729 Hypothesis 
nonplanar, 724-725 continuum, 175, 186 
nonseparable, 683 Hadamard, Jacques, 262 inductive, 313 
orientable, 740 Haken, Wolfgang, 728 Hypothesis of a conditional statement, 6 
paths in, 678-681 Half adder, 826, 843 Hypothetical syllogism, 71 
planar, 718-725, 736 Half subtractor, 828 
precedence, 647 Hall’s marriage theorem, 659 “TIcosian Game,”, 700 
protein interaction, 648 Hall, Philip, 659, 660 Icosian Puzzle, 698 
pseudograph, 643, 644, 735 Halting problem, 201, 895, 900 Idempotent laws 
radius of, 741 Hamilton’s “Voyage Around the World” Puzzle, 700 for Boolean algebra, 815, 819 
regular, 667, 736 Hamilton, Sir William Rowan, 698, 700 for lattices, 637 
representing, 668-675 Hamilton, William Rowan, 729 for propositions, 27 


self-complementary, 677 Hamilton circuits, 698-703, 736 for sets, 129 


Identification number 
single error in, 291 
Identification number(s) 
for airline tickets, 293 
Identification numbers 
for money orders, 293 
Identifier, 611 
ALGOL, 855 
C language, 857 
Identifying a sequence from its initial terms, 161 
Identities 
Boolean algebra, 27 
set, 129 
set(, 129 
set), 132 
Identities, combinatorial, 417-421 
Identities for Boolean algebra, 129 
Identity 
Bézout’s, 270 
combinatorial, proof of, 412, 439 
hexagon, 421 
proving, generating functions for, 548 
Vandermonde’s, 420-421 
Identity element(s) 
for integers modulo m, 243 
Identity elements axiom, A-1 
identity function, 144 
Identity laws, 27 
additive, A-1 
for Boolean algebra, 815, 816, 818 
for sets, 129 
multiplicative, A-1 
Identity matrix, 180, 186 
If—then construction, 8 
If then statement, 6 
If and only if, 9 
Tff, 9 
Image of an element, 139, 186 
Image of a set, 141 
Implicant, 832 
essential, 832 
prime, 832 
Implication, 6, 110 
Implicit use of biconditionals, 10 
In-degree of vertex, 654, 736 
Incidence matrices, 671, 736 
Incident edge, 651 
Inclusion—-exclusion principle, 128, 392-394, 
553-557, 566 
alternative form of, 558-559 
applications with, 558-564 
Inclusion relation, 619 
Inclusive or, 5 
Incomparable elements in poset, 619, 633 
Incomplete induction, 334 
Incorrect proof by mathematical induction, 757 
Increasing function, 143 
Increment 
or linear congruential method, 288 
Independence number, 741 
Independent events, 452, 453, 457-458, 494 
Independent random variables, 485-487, 494 
Independent set of vertices, 741 
Index of summation, 163 
Index registers, 732 
Indicator random variable, 492 
Indirect proofs, 83 
Indistinguishable 
boxes, 428-431 
objects, 428-431 
objects, permutations with, 428 
strings, 888 


Induction 
complete, 334 
generalized, 356-357 
incomplete, 334 
mathematical, 29, 311-329 
principle of, 377 
proofs by, 315-329 
second principle of, 334 
strong, 333-335, 378 
structural, 353-356, 378 
validity of, 326 
well-ordered, 620, 634 
Inductive definitions, 345-357 
of functions, 345-349, 378 
of sets, 349-356, 378 
of strings, 350 
of structures, 349-356 
Inductive hypothesis, 313 
Inductive loading, 333, 343, 380 
Inductive reasoning, 312 
Inductive step, 313, 377, 752, 768 
Inequality 
Bernoulli’s, 330 
Bonferroni’s, 467 
Boole’s, 467 
Chebyshev’s, 491, 495 
Markov’s, 493 
of harmonic numbers, 320-321 
proving by mathematical induction, 319-320 
triangle, 108 
Inference 
for program correctness, 373 
tule of, 110 
rules of, 69-71 
Infinite graph, 641 
Infinite ladder, 311 
by strong induction, 334 
Infinite series, 167 
Infinite set, 121 
Infinitude of primes, 260 
Infix form, 780 
Infix notation, 779-782, 804 
Influence graphs, 645 
Information flow, lattice model of, 627 
Initial assertion, 372, 378 
Initial conditions, 158, 565 
Initial position 
of a Turing machine, 889 
Initial state, 859 
finite-state automaton, 867 
of a Turing machine, 889 
Initial terms 
identifying a sequence using, 161 
Initial value, A-14 
Initial vertex, 594, 654 
Injection, 141, 186 
Injective (one-to-one) function 
counting, 387 
Injective function, 141 
Inorder traversal, 773, 775, 778, 804 
Input 
to an algorithm, 193 
Input alphabet, 859 
Input string 
finite-state machine, 861 
Insertion sort, 196, 197, 232 
average-case complexity of, 483-484 
worst-case complexity, 222 
Instantiation 
existential, 76 
universal, 75 
Integer 
composite, 257, 306 
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perfect, 272 

prime, 257 

signed, 855 
Integer(s) 

linear combination of, 306 
Integer-valued function, 140 
Integers 

axioms for, A-5 

division of, 238-240 

multiplication of 

fast, 528-529 

pairwise relatively prime, 306 

partition of, 359 

relatively prime, 306 

set of, 116 

squarefree, 564 
Integer sequences, 162 
Integers modulo m, 243 

additive inverses, 243 

associativity, 243 

closure, 243 

commutativity, 243 

distributivity, 243 

identity elements, 243 
Intension, of database, 585 
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Interconnection networks for parallel computation, 


661-663 
Interest, compound, 160 
Interior of simple polygon, 338 
Interior vertices, 603 
Internal vertices, 748, 804 


International Mathematical Olympiad (IMO), 263, 


299 


International Standard Book Number (ISBN), 291 
International Standard Serial Number (ISSN), 293 


Internet, search engines on, 794 
Internet addresses, counting, 392 
Internet datagram, 399 
Internet Movie Database, 645 
Internet Protocol (IP) multicasting, 786 
Intersection 

of fuzzy sets, 138 
Intersection graph, 650 
Intersection of a collection of sets, 133 
Intersection of multisets, 138 
Intersection of sets, 127, 185 
Interval(s), 117 

closed, 117 

open, 117, 332 
Intractable 

problems, 226 
Intractable problem, 232, 897 
Invariant for graph isomorphism, 672, 736 
Invariants, loop, 375-376, 378 
Inverse, 8 

modular, 275, 306 
Inverse, multiplicative, 60 
Inverse function, 145, 186 
Inverse law 

for addition, A-2 

for multiplication, A-2 
Inverse of a square matrix, 184 
Inverse relation, 582 
Inversions, in permutation, expected number of, 

482-484 

Inverter, 21, 823, 843 
invertible function, 145 
Invertible matrix, 184 
IP multicasting, 786 
Irrationality of 2, 342, 381 
Irrational number(s), 85, 116 
Irreflexive relation, 581 
ISBN-10, 291 
ISBN-13, 291 

check digit, 308 
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ISBN check digit, 291 
Isobutane, 750 
Isolated vertex, 652, 736 
Isomorphic graphs, 668, 671-675, 736 

paths and, 687-688 
Iterated function, 360 
Iterated logarithm, 360 
Iteration, 160, 360 

using to solve recurrence relations, 159 
Iteration using backward substitution, 160 
Iteration using forward substitution, 160 
Iterative algorithm, for Fibonacci numbers, 366 
Iterative procedure, for factorials, 366 
Iwaniec, Henryk, 264 


Jacobean rebellion, 151 
Jacquard loom, 31 
Jarnik, Vojtéch, 798 
Java, 855 
Jealous husband problem, 693 
Jersey crags, 163 
Jewish—Roman wars, 512 
Jigsaw puzzle, 342 
Job(s) 
assignment of, 565, 658 
lateness of, 235 
slackness of, 235 
Join, in lattice, 637 
Join of n-ary relations, 588 
Join of zero—one matrices, 181 
Joint authorship, 645 
Jordan curve theorem, 338 
Josephus, Flavius, 512 
Josephus problem, 512 
Jug, 109, 693 


k-connected graph, 684 
k-tuple graph coloring, 734 
K-maps, 830-836, 843 
K6nigsberg, 171 
KGnigsberg bridge problem, 693-694, 696, 697 
Kakutani’s problem, 107 
Kaliningrad, Russia, 693 
Karnaugh, 830-836 
Karnaugh, Maurice, 830 
Karnaugh maps, 830-836, 843 
Kayal, Neeraj, 262 
Kempe, Alfred Bray, 728 
Key, 295 

composite, 585 

encryption, 306 

for Caesar cipher, 295 

hashing function, 287 

primary, 585-586 

public, 299 
Key exchange, 302 
Key exchange protocol, 305, 306 
Keystream 

autokey cipher, 309 
King Hermeas, 2 
Kissing problem, 163 
Kleene’s theorem, 880, 900 
Kleene, Stephen Cole, 867, 878 
Kleene closure, 866, 899 
Knapsack problem, 235, 568 
Kneiphof Island, 693 
Knight’s tour, 707 

reentrant, 707 
Knights, knaves, and normals puzzles, 112 
Knights, knaves, and spies puzzles, 112 
Knights and knaves puzzles, 19 
Knuth, Donald, 196, 206 
Knuth, Donald E., 208 


Kruskal’s algorithm, 799-801, 804 
Kruskal, Joseph Bernard, 799, 800 
Kuratowski’s theorem, 724-725, 737 
Kuratowski, Kazimierz, 724 


L6b’s paradox, 112 

Labeled tree, 756 

Ladder, infinite, 311-334 

Lady Byron, Annabella Millbanke, 31 

Lagarias, Jeffrey, 107 

Lamé’s theorem, 347 

Lamé, Gabriel, 349 

Landau, Edmund, 206, 207 

Landau symbol, 206 

Language, 847-899 
context-free, 851 
context-sensitive, 886 
formal, 848 
generated by grammar, 850 
natural, 847-848 
of a grammar, 850 
recognized 


by nondeterministic finite-state automaton, 873 


recognized by an automaton, 899 


recognized by finite-state automata, 868 


regular, 851 
Language recognizer, 863 
Laplace’s definition of probability, 446 
Laplace, Pierre Simon, 445, 447, 472 
Large integers 

computer arithmetic with, 279 
Large numbers, law of, 459 
Lateness of a job, 235 
Lattice model of information flow, 627 
Lattice point, 380 
Lattices, 626-627, 634 

absorption laws for, 637 

associative laws for, 637 

bounded, 637 

commutative laws for, 637 

complemented, 637 

distributive, 637 


compatibility 

additive, A-2 

multiplicative, A-2 
completeness, A-2 
De Morgan’s 
or Boolean algebra, 815, 819 
or propositions, 28 
proving by mathematical induction, 323-324 
stributive, A-2 
or Boolean algebra, 815, 818 
‘or propositions, 27 
domination 
or Boolean algebra, 815 
idempotent 
or Boolean algebra, 815, 819 
or lattices, 637 
‘or propositions, 27 
identity 
additive, A-1 
or Boolean algebra, 815, 816, 818 
multiplicative, A-1 
inverse 
or addition, A-2 
or multiplication, A-2 
of detachment, 71 
of double complement, in Boolean algebra, 815, 
818 
of large numbers, 459 
of mathematical induction, A-5 
transitivity, A-2 
trichotomy, A-2 
used to prove basic facts, A-3—A-5 


di 


Law of total expectation, 492 
Laws 


De Morgan’s, 26 


Laws of propositional logic, 27 


absorption, 27 
associative, 27 
commutative, 27 
De Morgan’s, 27 
distributive, 27 
domination, 27 
double negation, 27 
identity, 27 


duality in, 639 
idempotent laws for, 637 
join in, 637 

meet in, 637 

modular, 639 


Law 


complement, 129 
complementation, 129 
Hardy—Weinberg, 97 


Law(s) 


absorption 

or Boolean algebra, 815, 816 
or lattices, 637 

or propositions, 27 
associative 

or addition, A-1 

or Boolean algebra, 815, 818 
or lattices, 637 

or multiplication, A-1 

or propositions, 27 

closure 

or addition, A-1 

or multiplication, A-1 
commutative 

or addition, A-1 

or Boolean algebra, 815, 818 
or lattices, 637 

or multiplication, A-1 


Laws of Thought, The (Boole), 472, 811 
Leaf, 746, 804 
Least common multiple, 266, 306 
Least element of poset, 625, 634 
Least upper bound, 634, A-2 
of poset, 625 
Left child of vertex, 749 
Left subtree, 749 
Lemma, 81, 110 
pumping, 888 
Length of bit string, 505 
Length of path 
in directed graph, 599 
in weighted graph, 708 
Length of string 
recursive definition of, 350 
“Less than or equals” relation, 619 
Letters of English 
relative frequency, 296 
Level of vertex, 753, 804 
Level order of vertex, 806 
Levin, Leonid, 227 
Lewis Carroll (C. L. Dodgson), 50 


Lexicographic ordering, 356, 435, 620-622 


Liber Abaci (Fibonacci), 348 
Library sort, 196 


Light fixture controlled by three switches, circuit for, 


825 
Limit, definition of, 61 


Linear array, 662 
Linear bounded automata, 886 
Linear combination of integers, 269, 306 
Linear complexity, 225 
Linear congruence, 275, 306 
systems of, 277-279 
Linear congruence(s) 
solving, 277 
Linear congruential method, 288 
increment, 288 
modulus, 288 
multiplier, 288 
seed, 288 
Linear homogenous recurrence relations, 514-520, 
565 
Linearithmic complexity, 225 
Linearity of expectations, 477-484, 494 
Linearly ordered set, 619, 633 
Linear nonhomogenous recurrence relations, 
520-524, 565 
Linear ordering, 619, 633 
Linear probing function, 288 
Linear search 
average-case time complexity, 221 
Linear search algorithm, 194, 232 
average-case complexity of, 482-484 
recursive, 363 
time complexity of, 220 
LISP, 855 
List of 23 problems 
David Hilbert, 176 
Lists, merging two, 368 
Literal, 820, 843 
Little-o notation, 218 
Littlewood, John E., 97 
Load balancing problem, 235 
Loading, inductive, 333, 343, 380 
Load of a processor, 235 
Loan, 169 
Lobsters, 524 
Local area networks, 661 
Logarithm 
discrete, 284 
Logarithm, iterated, 360 
Logarithm discrete, 306 
Logarithmic complexity, 225 
Logarithmic function, A-7—A-9 
logarithms 
big-O estimates of, 212 
Logic, | 
fuzzy, 16 
predicate, 37 
propositional, 3 
Logical connectives, 4 
Logical equivalences, 27 
constructing, 29 
Logical Expression(s) 
translating English sentences into, 62-63 
Logical expressions 
translating English sentences into, 16-17 
translating mathematical statements into, 60, 61 
Logically equivalent compound propositions, 25, 110 
Logically equivalent expressions, 110 
Logically equivalent statements, 45 
Logical operators, 109 
functionally complete, 35 
precedence of, 11 
Logic circuit, 20, 110 
Logic gates, 21, 110, 822-827 
AND, 21, 823, 843 
combination of, 823 
NAND, 828 
NOR, 828 


OR, 21,823, 843 
threshold, 845 
Logic programming, 51 
Logic puzzles, 19-20 
Long-distance telephone network, 646 
Longest common subsequence problem, 568 
Loom, Jacquard, 31 
Loop constructions, A-14—A-15 
Loop invariants, 375-376, 378 
Loops 
in directed graphs, 594, 633 
nested, 58 
within loops, A-15 
Lord Byron, 31 
Lottery, 447 
Mega Millions, 496 
Powerball, 496 
Lovelace, Countess of (Augusta Ada), 29, 31 
Lower bound 
of lattice, 637 
of poset, 625, 634 
Lower limit of a summation, 163 
Lucas, Edouard, 503 
Lucas, Francois Edouard, 162 
Lucas numbers, 379, 525 
Lucas sequence, 162 
Lucky numbers, 570 
Lukasiewicz, Jan, 780, 782 


m-ary tree, 748, 804 
complete, 756 
full, 748, 752 
height of, 754-755 
m-tuple, 586 
Ménages, probl‘eme des, 571 
Machine 
finite-state, 899 
Mealy, 863, 899 
Moore, 863, 865 
Turing, 899 
machine 
unit-delay, 861 
Machine(s) 
delay, 861-862 
finite-state, 847, 858-859 
with no output, 865 
with outputs, 859-863 
Turing, 847, 886, 888, 889, 893 
computing functions with, 892-893 
definition of, 889 
nondeterministic, 893 
sets recognized by, 891-892 
types of, 893 
Machine minimization, 872 
vending, 858-859 
MAD Magazine, 208 
Magic tricks, 20 
Majority voting, circuit for, 825 
Makespan, 235 
Making change 
greedy algorithm, 199 
Male optimal, 234 
Mappings, 139 
Maps 
coloring of, 727 
Markov’s inequality, 493 
markup languages, 858 
Massachusetts Institute of Technology (M.L.T.), 299 
Master theorem, 532 
Matching, 659 
maximum, 659 
stable, 204 
string, 231 
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Matchings with forbidden pairs, 234 
Mathematical induction, 29, 311-329 
Axiom of, A-5 
errors in, 328-329 
generalized, 356-357 
guidelines for, 328 
incorrect proof by, 757 
inductive loading with, 333 
principle of, 377 
proofs by, 315-329 
errors in, 328-329 
of divisibility facts, 321 
of inequalities, 319-320 
of results about algorithms, 324 
of results about sets, 323-324 
of summation formulae, 315 
second principle of, 334 
strong, 333-335, 378 
structural, 353-356, 378 
template for, 328 
template for proofs, 329 
validity of, 328 
Matrix (matrices) 
addition, 178 
adjacency, 669-671, 736 
counting paths between vertices by, 688-689 
entry of, 178 
equal, 178 
identity, 180, 186 
incidence, 671, 736 
invertible, 
multiplication 
algorithm for, 222 
complexity of, 223 
noncommutativity of, 179 
fast, 529 
representing relations using, 591-594 
row of a, 178 
sparse, 670 
square, 178 
symmetric, 181, 186 
transpose of, 181, 186 
upper triangular, 231 
zero—one, 181, 186 
of transitive closure, 602-603 
representing relations using, 591-594 
Warshall’s algorithm and, 604-606 
Matrix-chain multiplication, 223 
Matrix-chain multiplication problem, 513 
Matrix product, 179 
Maurolico, Francesco, 313 
Maximal element of poset, 624, 634 
Maximum, of sequence, 528 
Maximum element 
algorithm for finding, 193 
in finite sequence, 192 
Maximum matching, 659 
Maximum satisfiability problems, 498 
Maximum spanning tree, 802 
Maxterm, 822 
McCarthy 
function, 380 
McCarthy, John, 381 
McCluskey, Edward J., 837 
Mealy, G. H., 863 
Mealy machine, 863, 899 
Mean, 203 
arithmetic, 100, 332 
deviation from, 491 
geometric, 100, 332 
harmonic, 108 
quadratic, 108 
Median, 203 
Meet, in lattice, 637 
Meet of zero—one matrices, 181 
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Mega Millions, Lottery, 496 
Meigu, Guan, 698 
Member of a set, 185 
Membership table, 186 
Members of a set, 116 
memoization, 510 
Merge sort, 196, 367-370, 378, 528 
complexity of, 532 
recursive, 368 
Merging two lists, 368 
Mersenne, Marin, 261 
Mersenne primes, 261, 306 
Mesh network, 662-663 
Mesh of trees, 809 
metacharacters, 858 
Metafont, 208 
Method 
middle-square, 292 
roster, 116 
Method(s) 
Critical Path, 639 
Horner’s, 230 
probabilistic, 465-466, 494 
Quine—McCluskey, 830, 837-840 
Method(s) of proof, 82 
by cases, 92, 95 
by contradiction, 86 
by contraposition, 83 
by exhaustion, 93 
direct, 82 
exhaustive, 93 
proofs of equivalence, 87 
trivial, 84 
vacuous, 84 
Method roster, 185 
Middle-square method, 292 
Millennium Prize problems, 227 
Miller’s test, 286 
Miller’s test for base b, 286 
Minimal element of poset, 624, 634 
Minimization 
of Boolean functions, 828-840, 843 
of combinational circuits, 828-840 
Minimization of a finite-state machine, 872 
Minimizing maximum lateness 
greedy algorithm for, 235 
Minimum, of sequence, 528 
Minimum dominating set, 739 
Minimum spanning forest, 802 
Minimum spanning trees, 797-802, 804 
Minmax strategy, 767, 804 
Minterm, 820, 843 
Mistakes in proofs, 89-90 
Mixed graph, 644 
mod function, 239, 306 
Mode, 203 
Modeling 
computation, 847-897 
with graphs, 644-649 
with recurrence relations, 502-507 
with trees, 749-752 
Modular arithmetic, 240-306 
Modular exponentiation, 253 
algorithm for, 253 
recursive, 363 
Modular inverse, 275, 306 
Modular lattice, 639 
Modular properties, in Boolean algebra, 819 
Module dependency graphs, 647 
Modulus 
for linear congruential method, 288 
Modus ponens, 71 
universal, 77 


Modus tollens, 71 

universal, 77 
Mohammed’s scimitars, 697 
Molecules, trees modeling, 750 
Money orders 

identification numbers for, 293 
Monoalphabetic cipher, 297 
Monotone decreasing property of graph, 742 
Monotone increasing property of graph, 742 
Monotonic grammars, 852 
Monte Carlo algorithms, 463-465 
Montmort, Pierre Raymond de, 563, 571 
Monty Hall Three Door Puzzle, 450, 452, 476, 499 
Monty Python, 472 
Moore, E. F., 863 
Moore machine, 863, 865 
Moth, 872 
Motorized pogo stick, 812 
Mr. Fix-It, 263 
Muddy children puzzle, 19 
Multicasting, 786 
Multigraphs, 642, 644, 735 

Euler circuit of, 697 

Euler path of, 697 

undirected, 644 
Multinomial coefficient, 434 
Multinomial theorem, 434 
Multiple, 238 

least common, 266, 306 
Multiple edges, 642-644, 735 
Multiple output circuit, 826 
Multiplexer, 828 
Multiplication 

of function, 141 

matrix-chain, 223 

of integers 

fast, 528-529 
of matrices 
fast, 529 

Multiplication algorithm, 251 
Multiplicative Compatibility Law, A-2 
Multiplicative inverse, 60 
Multiplicities of elements in a multiset, 138 
Multiplier 

for linear congruential method, 288 
Multiset, 138 

multiplicities of elements, 138 
Multisets 

difference of, 138 

intersection of, 138 
Mutually independent events, 497 
Mutually independent trials, 458 


n-ary relations, 583-589, 633 
domain of, 584 
operations on, 586-588 

n-cubes, 655 

n-queens problem, 792-793 

n-regular graph, 667 

n-tuple 
ordered, 122 

n-tuples, 584-585 

Naive set theory, 118 

Namagiri, 98 

NAND, 821, 843 

NAND gate, 828 

Natural language, 847-848 

Natural numbers 
set of, 116 

Naur, Peter, 854 


NAUTY, 674 
Naval Ordnance Laboratory, 872 
Navy WAVES, 872 
Necessary and sufficient conditions, 9 
Necessary condition, 6 
expressing a conditional statement using, 6 
Necessary for, 6 
Negating conjunctions, 28 
Negating disjunctions, 28 
Negating Quantified Expressions, 46 
Negation, 109 
of a proposition, 3 
of nested quantifiers, 63-64 
truth table for, 4 
Negation operator, 4 
Negative 
false, 471 
true, 471 
Neighbors in graphs, 651 
Neptune, 476 
Nested loops, 58 
Nested quantifiers, 57-64 
negating, 63-64 
Network(s) 
computer, 641 
interconnection networks, 661-663 
local area networks, 661 
multicasting over, 786 
with diagnostic lines, 642 
with multiple lines, 642 
with multiple one-way lines, 642 
with one-way lines, 643 
gating, 822-823 
depth of, 828 
examples of, 823-825 
minimization of, 828-840 
social, 644 
tree-connected, 751-752 
Network number (netid), 392 
Newton-Pepys problem, 500 
Niche overlap graph, 648 
Nickels, 199 
Nim, 766, 768 
Nobel Prize, 119 
Nodes, 594, 641 
Noncommutativity of matrix multiplication, 179 
Nonconformists, 472 
Nonconstructive existence proof, 96, 110 
Noncontracting grammar, 851 
Nondeterministic finite-state automaton, 873, 899 
equivalent finite-state automaton, 874 
language recognized by, 873 
Nondeterministic polynomial-time problems, 227 
class of, 900 
item Nondeterministic Turing machine, 893, 899 
Nonoverlapping ears, 343 
Nonplanar graphs, 724-725 
Nonregular set, 891 
recognition by Turing machine, 891 
Nonresidue 
quadratic, 286 
Nonseparable graph, 683 
Nonterminals, 849 
NOR, 821, 843 
NOR gate, 828 
Normal form 
disjunction, 35 
prenex, 68 
NOT, 18 


Notation 
big-O, 205, 232 
big-Omega (Q), 214, 232 
big-Theta, 215 
big-Theta (©), 232 
dependency, 846 
for products 
well-formed formula in, 784 
infix, 779-782, 804 
little-o, 218 
Polish, 780, 804 
postfix, 779-782, 804, 858 
prefix, 779-782, 804 
product, 186 
reverse Polish, 781, 804, 858 
set builder, 116, 185 
summation, 162, 186 
NOT gate, 21 
Noun, 848 
Noun phrase, 848 
Nova, 107 
NP, class of nondeterministic polynomial-time 
problems, 900 
NP-complete problems, 227, 715, 830, 900 
Null quantification, 56 
Null set, 185 
Null string, 849 
Number(s) 
Bacon, 680 
Bell, 618 
cardinal, 121 
Carmichael, 283, 306 
Catalan, 507 
chromatic, 728, 737 
edge, 734 
crossing, 726 
computable, 886 
Cunningham, 262 
Erdés, 635, 680, 689 
Fibonacci, 347, 365-367, 771 
formula for, 517 
iterative algorithm for, 366 
rabbits and, 502-503 
recursive algorithms for, 365 
harmonic 
inequality of, 320-321 
independence, 741 
irrational, 85, 116 
large, law of, 459 
Lucas, 379, 525 
lucky, 570 
natural, 116 
pseudorandom, 288 
Ramsey, 404 
rational, 85, 116 
real, A-1—A-5 
Stirling, of the first kind, 431 
signless, 443 
Stirling, of the second kind, 430 
Ten Most Wanted, 262 
Ulam, 188 
Number-theoretic functions, 892 
Numbering plan, 387 
Number theory, 237 
definition of, 237 


Object, 118 
Object(s) 
distinguishable, 428 
and distinguishable boxes, 428-429 
and indistinguishable boxes, 428-431 


indistinguishable, 428 
and indistinguishable boxes, 428-431 
unlabeled, 428 
Obligato game, 112 
Octahedral die, 496 
Octal expansions, 246, 306 
Octal representation, 246, 306 
Odd, 83 
Odd pie fights, 325 
Odlyzko, Andrew, 163 
Odometer, 307 
One’s complement representations, 256 
One-to-one (injective) function 
counting, 387 
One-to-one correspondence, 144, 186 
One-to-one function, 141, 143, 186 
On-Line Encyclopedia of Integer Sequences (OEIS), 
162 
Only if, expressing conditional statement using, 7 
Onto (surjective) function, 143, 186 
number of, 560-562, 566 
Open interval, 117, 332 
Open problems, 106, 263-264 
Operands, well-formed formula of, 351 
Operation(s) 
bit, 11, 110 
bitwise, 110 
set, 127 
Operations 
on n-ary relations, 586-588 
Operator(s) 
bitwise, 12 
logical, 109 
negation, 4 
logical, 109 
selection, 586 
well-formed formula of, 351 
Opium, 475 
Optimal algorithm, 230 
Optimal for suitors, stable assignment, 343 
Optimal solution, 198 
Optimization problems, 198 
OR, 18 
Or 
exclusive, 5, 6 
inclusive, 5 
Oracle of Bacon, 680 
Order, 207 
of quantifiers, 58 
Ordered n-tuple, 122 
Ordered rooted tree, 749, 804, 806 
Ordering 
dictionary, 435 
lexicographic, 356, 435, 620-622 
linear, 619 
partial, 618-629, 633 
quasi-ordering, 637 
total, 619, 633 
Ordered pair 
defined using sets, 126 
Ordinary generating function, 537 
Ore’s theorem, 701, 707 
Ore, O., 701 
Organizational tree, 750 
OR gate, 21, 823, 843 
Orientable graphs, 740 
Orientation of undirected graph, 740 
Out-degree of vertex, 654, 736 
Output 
to an algorithm, 193 
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Output alphabet, 859 
Outputs 
finite-state machines with, 859-863 
finite-state machines without, 865 
Output string 
finite-state machine, 861 


P(n,r), 439 
P, class of polynomial-time problems, 900 
P=NP problem, 897 
Pair, devil’s, 678 
Pairs, forbidden, 234 
Pairwise relatively prime integers, 306 
Palindrome(s), 202, 397, 857 
set of, 888 
Paradigm 
Algorithmic, 224, 232 
Paradox, 118, 185 
barber, 16 
Lob’s, 112 
Russell’s, 126 
St. Petersburg, 497 
Parallel algorithms, 661 
Parallel edges, 642 
Parallel processing, 229, 661 
tree-connected, 751-752 
Parentheses, balanced strings of, 382 
Parent of vertex, 747, 803 
Parent relation, 580 
Parity 
same, 83 
Parity check bit, 290 
Parse tree, 852-854, 899 
Parsing 
bottom-up, 853 
top-down, 853 
Partial correctness, 372 
Partial function, 152, 186, 889 
codomain of, 152 
domain of, 152 
domain of definition, 152 
undefined values, 152 
Partially ordered set, 618 
antichain in, 637 
chain in, 637 
comparable elements in, 622, 633 
dense, 632 
dual of, 630 
greatest element of, 625, 634 
Hasse diagram of, 622-624, 634 
incomparable elements in, 622, 633 
least element of, 625, 634 
lower bound of, 625, 634 
maximal element of, 624, 634 
minimal element of, 624, 625, 634 
upper bound of, 625, 634 
well-founded, 632 
Partial orderings, 618-629, 633 
compatible total ordering from, 634 
Partition, 552 
of positive integer, 359, 431 
of set, 612-614, 633 
refinement of, 617 
Partner 
valid, 234 
Pascal’s identity, 418-419, 439 
Pascal’s triangle, 419, 439 
Pascal, Blaise, 282, 419, 445, 452 
Passwords, 66, 391 
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Paths, 678-681 
and graph isomorphism, 687-688 
counting between vertices, 688-689 
Euler, 693-698, 736 
Hamilton, 698-703, 736 
in acquantanceship graphs, 680 
in collaboration graphs, 680 
in directed graphs, 599-600, 633, 736 
in directed multigraphs, 679 
in Hollywood graph, 680-68 1 
in simple graphs, 679 
in undirected graphs, 679 
length of, in weighted graph, 708 
of length n, 680 
shortest, 707-716, 736 
terminology of, 679 
Payoff, 765 
Pearl Harbor, 872 
Pecking order, 741 
Peirce, Charles Sanders, 38 
Peirce arrow, 36 
Pele, A., 536 
Pendant vertex, 652, 736 
Pennies, 199 
Perfect integer, 272 
Perfect power, 93 
Perfect square, 83 
Peripatetics, 2 
Permutation, circular, 415 
Permutations, 407—409, 439 
generating, 434-436 
generating random, 499 
inversions in, expected number of, 482-484 
with indistinguishable objects, 427-428 
with repetition, 423 
PERT (Program Evaluation and Review Technique), 
639 
Petersen, Julius Peter Christian, 706 
Phrase-structure grammar, 849, 899 
Phrase-structure grammars, 849-854 
Pick’s theorem, 342 
Pie fights, odd, 325 
Pigeonhole principle, 86, 399-405, 439 
applications with, 403-405 
generalized, 401-403, 439 
Planar graphs, 718-725, 736 
Plato, 2 
Plato’s Academy, 2, 259 
PNF (prenex normal form, 68 
Pogo stick, motorized, 812 
Pointer, position of, digital representation of, 
702-703 
Poisonous snakes, 763 
Poker hands, 411 
Polish notation, 780, 804 
Polygon, 338 
convex, 338 
diagonal of, 338 
exterior of, 338 
interior of, 338 
nonconvex, 338 
sides of, 338 
simple, 338 
vertices of, 338 
with nonoverlapping ears, 343 
Polynomial-time algorithm for primality, 262 
Polynomial-time problems 
class of, 900 
Polynomial complexity, 225 
Polynomials 
big-O estimates for, 212 
big-O estimates of, 209 
big-Theta estimates for, 216 


Polynomials, rook, 571 
Polyominoes, 105, 326 
Ponens 
modus, 71 
Population of the world, 168 
Poset, 618, 633 
antichain in, 637 
chain in, 637 
comparable elements in, 619, 633 
dense, 632 
dual of, 630 
greatest element of, 625, 634 
Hasse diagram of, 622-624, 634 
incomparable elements in, 619, 633 
least element of, 625, 634 
lower bound of, 625, 634 
maximal element of, 624, 634 
minimal element of, 624, 625, 634 
upper bound of, 625, 634 
well-founded, 632 
Positive 
false, 471 
true, 471 
Positive integers 
axioms for, A-5 
Postcondition, 372 
Postconditions, 39 
Postfix form, 781 
Postfix notation, 779-782, 804, 858 
Postorder traversal, 773, 776-778, 804 
Postulates, 81 
Potrzebie System of Weights and Measures, 208 
Powerball, Lottery, 496 
Power generator, 292 
Powers 
of relation, 580-581, 633 
Power series, 538-541 
formal, 538 
Power set, 121, 185 
Pre-image of an element, 139, 186 
Precedence 
of logical operators, 11 
of quantifiers, 44 
Precedence graphs, 647 
Precondition(s), 39, 372 
Predicate, 110 
truth set of, 125 
Predicate calculus, 40 
Predicate logic, 37 
Prefix codes, 762-764, 804 
Prefix form, 780 
Prefix notation, 779-782, 804 
well-formed formula in, 784 
Premise of a condition statement, 6, 110 
Premises of an argument, 69, 70 
Prenex normal form (PNF), 68 
Preorder traversal, 773-775, 778, 804 
Prim’s algorithm, 798-799, 804 
Prim, Robert Clay, 798, 800 
Primality testing, 464-465 
Primary key, 585-586, 633 
Prime, 257, 306 
Mersenne, 306 
primitive root of a, 306 
probability positive integer less than n is, 262 
Prime(s) 
arithmetic progression of, 263 
conjectures about, 263 
distribution of, 261 
in arithmetic progressions, 262 
infinitude of, 260 
Mersenne, 261, 306 
of form n? + 1, 264 


primitive root of a, 306 
probability positive integer less than n is, 262 
twin, 264 
Prime factorization, 259 
Prime implicant, 832, 843 
essential, 832 
Prime number theorem, 262, 636 
Primitive root of a prime, 284, 306 
Prince of Mathematics, 241 
Princess of Parallelograms, 31 
Principia Mathematica (Whitehead and Russell), 34 
Principia Mathematica, 119 
Principle(s) 
duality, for Boolean identities, 816 
of buoyancy, A-4 
of counting, 385-390 
of inclusion—exclusion, 392-394, 553-557, 566 
alternative form of, 559 
applications with, 558-564 
of mathematical induction, 377 
of well-founded induction, 635 
of well-ordered induction, 620 
pigeonhole, 86, 399-405, 439 
applications with, 403-405 
generalized, 401-403, 439 
Principle of inclusion—exclusion, 128 
Private key 
cryptosystem, 298, 306 
RSA system, 301 
Prize, Nobel, 119 
Probabilistic algorithms, 445, 463-465, 494 
Probabilistic method, 465-466, 494 
Probabilistic primality testing, 464-465 
Probabilistic reasoning, 450 
Probability, discrete, 445-500 
assigning, 453-455 
conditional, 453, 456-457, 494 
finite, 445-448 
in medical test results, 471-472 
Laplace’s definition of, 446 
of collision in hashing functions, 462-463 
of combinations of events, 449-450, 455-456 
Probability distribution, 453 
Probability generating function, 552 
Probability positive integer less than n is prime, 262 
Probability theory, 445, 452-466 
Probing function, 288 
Problem(s) 
n-queens, 792-793 
art gallery, 735 
birthday, 461-463 
bridge, 693-694, 696, 697 
celebrity, 332 
Chinese postman, 698 
class of NP-complete, 897 
class P, 896 
closest-pair, 532-535 
Collatz, 107 
decidable, 895 
decision, 894, 900 
discrete logarithm, 284 
halting, 201, 895, 900 
hatcheck, 481, 562 
Hilbert’s 23, 895 
Hilbert’s Tenth Problem, 895 
intractable, 226, 232, 897 
jealous husband, 693 
Josephus, 512 
Kakutani’s, 107 
kissing, 163 
knapsack, 235, 568 
load balancing, 235 
longest common subsequence, 568 
matrix-chain multiplication, 513 
maximum satisfiability, 498 


Millennium Prize, 227 
Newton—Pepys, 500 
NP, 227, 896 
NP-complete, 227, 715, 830, 900 
P, 227 
P=NP, 897 
P versus NP, 227 
open, 106, 263-264 
optimization, 198 
problem, two children, 498 
satisfiability, 34, 227 
Satisfiability 
solving, 34 
scheduling greedy algorithm for 
final exams, 731, 732 
searching, 194 
shortest-path, 707-716, 736 
solvable, 226, 232, 895, 900 
Syracuse, 107 
tiling, 895 
tractable, 226, 232, 897 
traveling salesman, 714-716, 736 
traveling salesperson, 702, 709 
two children, 498 
Ulam’s, 107 
undecidable, 895 
unsolvable, 226, 232, 895, 900 
utilities-and-houses, 718 
yes-or-no, 894 
Probl‘eme de rencontres, 563, 571 
Probl‘eme des ménages, 571 
Procedure statements, A-11 
Processing 
concurrent, 647 
parallel, 229, 661 
tree-connected, 751-752 
Product 
Boolean, 811-812, 821, 843 
matrix, 179 
Product-of-sums expansion, 820 
Production, 899 
Productions, 849 
of grammar, 849 
Product notation, 186 
Product rule, 386 
Program correctness, 372-378 
conditional statements for, 373-375 
loop invariants for, 375-376 
partial, 372 
program verification for, 372-373 
tules of inference for, 373 


Program Evaluation and Review Technique (PERT), 
639 
Programming 
logic, 51 
Programming, dynamic, 507 
Programming languages, 398, 611 
Program verification, 372-373 
Progression 
arithmetic, 186 
geometric, 157, 186 
sums of, 318-319 
Projection of n-ary relations, 586, 633 
Prolog, 51, 74 
Proof(s), 81, 110 
adapting, 101 
by cases, 92-110 
common errors with, 95 
by contradiction, 86, 110 
by contraposition, 83, 110 
by exhaustion, 93 
by mathematical induction, 315-329 
by structural induction, 353 


combinatorial, 412, 439 

constructive existence, 96 

direct, 82, 1180 

existence, 96 

indirect, 83 

mistakes in, 89 

nonconstructive existence, 96 

of equivalence, 87 

of recursive algorithms, 364-367 

trivial, 84, 110 

uniqueness, 99, 110 

vacuous, 84, 110 
Proof strategy, 85, 100-106 
Proper subset, 185 
Properties 

of algorithms, 193 
Property 

Archimedean, A-5 

Completeness, A-2 

Well-ordering, 340-341, 378, A-5 
Proposition(s), 2, 81, 109 

compound, 3, 25, 27 109 

negation of, 3, 109 
Propositional equivalences, 129 
Propositional calculus, 3 
Propositional function, 110 
Propositional logic, 3 

applications of, 16 

argument, 70 

rules of inference for, 71 
Propositional variable(s), 2, 109 
Protein interaction graph, 648 
Protestant Noncomformists, 472 
Protocol(s) 

cryptographic, 302 

Diffie-Helman key agreement, 302 

key exchange, 305, 306 
Pseudocode, 192, A-11—-A-15 
Pseudograph, 643, 644, 735 
Pseudoprime, 282, 306 

strong, 286 

to the base b, 282 
Pseudorandom numbers, 288 


generated by linear congruential method, 288 


middle-square generator, 292 
power generator for, 292 
pure multiplicative generator of, 289 
Public key 
cryptosystem, 298 
encryption, 306 
RSA system, 299 
Pumping lemma, 888 
Pure multiplicative generator, 289 
Pushdown automaton, 886 
Puzzle 
“Voyage Around the World,”, 700 
Birthday Problem, 461-463 
Icosian, 698 
jigsaw, 342 
knights, knaves, and normals, 112 
knights, knaves, and spies, 112 
knights and knaves, 19 
the lady or the tiger, 24 
logic(, 19 
logic), 20 
Monty Hall Three Door, 450, 452, 476, 499 
muddy children, 19 
Reve’s, 504 
river crossing, 692 
Sudoku, 32 
Sun-Tsu’s, 277 
Tower of Hanoi, 503-504 
zebra, 24 
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P versus NP problem, 227 
Pythagorean triples, 106 
Panini, 854 


Quadratic congruence, 285 
Quadratic mean, 108 
Quadratic nonresidue, 286 
Quadratic residue, 286 
Quad trees, 809 
Quality control, 463 
Quantification, 40 

as loops, 58 

existential, 42, 110 

null, 56 

universal, 110 
Quantification, universal, 40 
Quantified Expressions 

negating, 46 
Quantified statement(s) 

restricted domain, 124 

rules of inference for, 75 

rules of inference for), 77 
Quantifier 

existential, 42 

scope of, 45, 110 

uniqueness, 44 

universal, 40 
Quantifiers 

De Morgan’s law, 47 

nested 

negating, 63-64 

nested(, 57 

nested), 64 

order of, 58 

precedence of, 44 

use in system specifications, 50 
Quarters, 199 
Quasi-ordering, 637 
Queen of mathematics, 241 
Queen of the sciences, 241 
Queens on chessboard, 740 
Question, begging the, 90, 110 
Quick sort, 196, 371 
Quine, Willard van Orman, 837, 839 
Quine—McCluskey method, 830, 837-840 
Quotient, 238, 239, A-6 
Quotient automaton, 878 


r-combination, 410, 439 
r-permutation, 407, 439 
Rabbits, 502-503 
Races, horse, 31 
Radius of graph, 741 
Rado, Tibor, 899 
Ramanujan, Srinivasa, 97, 98 
Ramaré, O., 264 
Ramsey, Frank Plumpton, 404 
Ramsey number, 404 
Ramsey theory, 404 
Random permutation, generating, 499 
Random simple graph, 742 
Random variables, 454, 460-461 
covariance of, 494 
definition of, 494 
distribution of, 460, 494 
geometric, 484-485 
expected values of, 477-480, 491, 494 
independent, 485-487, 494 
indicator, 492 
standard deviation of, 487 
variance of, 477-480, 494 
Range of a function, 139, 186 
Ratio 
common, 157 
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Rational number(s), 85, 116 
countability of, 172 
Reachable, 807 
Reachable state, 901 
Real-valued function, 140 
Real number 
decimal expansion of a, 174 
Real numbers 
constructing, A-5 
least upper bound, A-2 
set of, 116 
upper bound, A-2 
Reasoning 
circular, 90, 110 
deductive, 312 
forward, 100 
inductive, 312 
probabilistic, 450 
Recognized language, 868 
Recognized strings, 868 
Recognizer 
language, 863 
Rencontres, 563, 571 
Records, 584 
Recurrence relation(s), 158, 186, 501-510 
associated homogenous, 520 
definition of, 158, 565 
divide-and-conquer, 527-535 
initial condition for, 158 
linear homogenous, 514-520, 565 
linear nonhomogenous, 520-524, 565 
modeling with, 502-507 
simultaneous, 526 
solution of, 158 
solving, 514-524 
generating functions for, 546-548 
solving using iteration, 159 
Recursion, 344 
Recursive algorithms, 360-370, 378 
correctness of, 364 
for binary search, 363 
for computing a”, 361 
for computing greatest common divisor, 362 
for factorials, 361 
for Fibonacci numbers, 365 
for linear search, 363 
for modular exponentiation, 363 
proving correct, 364-367 
trace of, 361, 362 
Recursive definitions, 311, 345-357 
of extended binary trees, 352 
of factorials, 361 
of functions, 345-349, 378 
of a sequence, 158 
of sets, 349-356, 378 
of strings, 349 
of structures, 349-356 
Recursive merge sort, 368 
Recursive modular exponentiation, 363 
Recursive sequential search algorithm, 363 
Recursive step, 349 
Reentrant knight’s tour, 707 
Refinement, of partition, 617 
Reflexive closure of relation, 598, 634 
Reflexive relation, 576, 633 
representing 
using digraphs, 594-596 
using matrices, 591 


Regions of planar representation graphs, 719, 736 


Regular expression, 899 

Regular expressions, 879 

Regular grammar(s), 851, 884, 899, 900 
Regular graph, 667 


Regular language, 851 
Regular set(s), 878-879, 884, 899, 900 
constructing automaton to recognize, 881 
Relation, 124 
recurrence, 158, 186 
Relation(s), 573-639 
n-ary, 583-589, 633 
domain of, 584 
operations on, 586-588 
“greater than or equal,”, 618-619 
“less than or equals,”, 619 
antisymmetric, 577-578, 633 
asymmetric, 582 
binary, 573, 633 
circular, 635 
closures of, 597-606, 633 
reflexive, 598, 633 
symmetric, 598, 634 
transitive, 597, 600-603, 634 
combining, 579-580 
complementary, 582 
composite of, 580, 633 
connectivity, 600, 633 
counting, 578-579 
covering, 623, 631 
diagonal, 598 
divide-and-conquer recurrence, 527 
divisibility, 619 
domains of, 584 
equivalence, 607-614, 633 
functions as, 574 
inclusion, 619 
inverse, 582, 633 
irreflexive, 581 
on set, 575-576 
parent, 580 
paths in, 599-600 
powers of, 580-633 
properties of, 576-579 
reflexive, 576, 633 
representing 
using digraphs, 594-596 
using matrices, 591-594 
symmetric, 577-578, 634 
transitive, 578-580, 633 
Relational database model, 584-586, 633 
Relation on a set, 124 
Relative frequency 
letters of English, 296 
Relatively prime integers, 306 
Remainder, 239, 306 
Removing and edge from a graph, 663 
Removing an edge from a vertices, 664 
Repetition 
combinations with, 424-427 
permutations with, 423 
Replacement 
sampling with, 448 
sampling without, 448 
Representation 
base b, 306 
binary, 306 
octal, 306 
unary, 892 
Representation hexadecimal, 306 
Representations 
one’s complement, 256 
two’s complement, 256 
Representative, of equivalence class, 610 
Residue 
quadratic, 286 
Resolution, 74 
Resolution (rule of inference), 71 


Restricted domain 
of quantified statement, 124 
quantifier with, 44 
return statement, A-15 
Reve’s puzzle, 504 
Reverse-delete algorithm, 803 
Reverse Polish notation, 781, 804 
reverse Polish notation, 858 
Right child of vertex, 749 
Right subtree, 749 
Right triominoes, 326, 333 
Ring commutative, 244 
Ring topology for local area network, 661 
River crossing puzzle, 692 
Rivest 
Ronald, 300 
Rivest, Ronald, 299 
RNA (ribonucleic acid), 388 
RNA chain sequencing, 443 
Roadmaps, 647 
Rock climbing, 163 
Rocket-powered Frisbee, 812 
Rook polynomials, 571 
Root 
primitive, 284 
Rooted Fibonacci trees, 757 
Rooted spanning tree, 797 
Rooted trees, 351, 747-749 
Sz-tree, 806 
B-tree of degree k, 805 
balanced, 753, 804 
binomial, 805 
decision trees, 760-762 
definition of, 803 
height of, 753, 804 
level order of vertices of, 806 
ordered, 749, 804, 806 
Roots, characteristic, 515 
Roster method, 116, 185 
Round-robin tournaments, 341, 649 
Routing transit number (RTN), 308 
check digit, 308 
Row of a matrix, 178 
Roy, Bernard, 603 
Roy—Warshall algorithm, 603-606 
RSA system, 299 
cryptosystem, 299, 301, 306 
decryption, 300 
decryption key, 301 
digital signatures, 303 
encryption 299 
private key, 301 
public key, 301 
Rule 
division, 394 
product, 386 
subtraction, 393 
sum, 386, 389, 439 
Rule(s) of inference, 69-71, 110 
addition, 71 
building arguments using, 73 
conjunction, 71 
disjunctive syllogism, 71 
for program correctness, 373 
for propositional logic, 71 
for quantified statements, 75-77 
hypothetical syllogism, 71 
modus ponens, 71 
modus tollens, 71 
resolution, 71, 74 
simplification, 71 
Run, 492 
Russell’s paradox, 126 
Russell, Bertrand, 118, 119 


Sx-tree, 806 
Same parity, 83 
Samos, Michael, 533 
Sample space, 446 
Sampling 
without replacement, 448 
with replacement, 448 
Sandwich, 858 
Sanskrit, 854 
Satisfiability, 30 
Satisfiability problem, 227 
solving, 34 
modeling Sudoku puzzle as, 32 
Satisfiability problem, maximum, 498 
Satisfiable compound proposition, 30, 110 
Saturated hydrocarbons, trees modeling, 750 
Saxena, Nitin, 262 
Scheduling problems 
final exams, 731, 732 
greedy algorithm for, 324-325 
software projects, 633 
talks, 200 
tasks, 629 
Schréder, Ernst, 175 
Schréder—Bernstein theorem, 174 
Scimitars, Mohammed’s, 697 
Scope of a quantifier, 45, 110 
Screw, Archimedes, A-4 
Search 
binary, 194 
linear, 194 
sequential, 194 
Search engines, 794 
Searches 
Boolean, 18 
Searching algorithms, 194-196, 232 
binary, 528 
breadth-first, 789-791, 804 
depth-first, 787-789, 804 
applications with, 794-795 
in directed graphs, 794-795 
linear 
average-case complexity of, 482-484 
recursive, 363 
recursive binary, 363 
recursive linear, 363 
recursive sequential, 363 
Searching problems, 194 
Search trees, binary, 757-759, 804 
Searcing 
web pages, 18 
Second principle of mathematical induction, 334 
Secret key 
exchange, 302 
Seed 
for linear congruential method, 288 
Sees, 735 
Selection operator, 586, 633 
Selection sort, 196, 203 
Self-complementary graph, 677 
Self-converse directed graph, 740 
Self-dual, 844 
Self-generating sequences, 382 
Semantics, 847 
Sentence, 848, 849 
Separating set, 684 
Sequence(s), 156, 186 
de Bruijn, 744 
Fibonacci, 158 
finding maximum and minimum of, 528 
generating functions for, 537 


integer, 162 
Lucas, 162 
recursive definition of, 158 
self-generating, 382 
strictly decreasing, 403 
strictly increasing, 403 
unimodal, 568 
Sequencing 
RNA chains, 443 
Sequential search algorithm, 194 
Serial algorithms, 661 
Series 
geometric, 164 
harmonic, 321 
infinite, 167 
power, 538-541 
formal, 538 
Set(s), 185 
Cartesian product of, 123 
complement of, 128, 129, 186 
computer representation of, 134 
countable, 171, 186 
cut, 806 
difference of, 128, 186 
disjoint, 128 
dominating, 739 
element of, 185 
elements of a, 116 
empty, 118, 185 
finite, 121, 185, 553, 565 
combinations of, 437-438 
counting of subsets, 388 
number of subsets of, 323 


union of three, number of elements in, 554-556, 


566 
union of two, number of elements in, 553, 565 
fuzzy, 138 
guarding, 735 
image of, 141 
infinite, 121 
linearly ordered, 619, 633 
member of, 185 
members of a, 116 
nonregular, 891 
not recognized by finite-set automata, 885 
null, 185 
of complex numbers, 116 
of irrational numbers, 116 
of natural numbers, 116 
of palindromes, 888 
of rational numbers, 116 
of real numbers, 116 
partially ordered, 618, 633 
antichain in, 637 
chain in, 637 
comparable elements in, 619, 633 
dense, 632 
dual of, 630 
greatest element of, 625, 634 
Hasse diagram of, 622-624, 634 
incomparable elements in, 619, 622, 633 
least element of, 625, 634 
lower bound of, 625, 634 
maximal element of, 624, 634 
minimal element of, 624, 625, 634 
upper bound of, 625, 634 
well-founded, 632 
partition of, 612-613, 633 
power, 121, 185 
proofs of facts about, by mathematical induction, 
321 
recognized by Turing machines, 891-892 
recursively defined, 349-356, 378 
regular, 878, 879, 884, 899 
relation on, 575-576 
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relation on, 124 
representing with a bit string, 134 
separating, 684 
singleton, 118 
successor of, 137 
symmetric difference of, 186 
totally ordered, 619, 633 
uncountable, 186 
union of, 127, 185 
universal, 118, 185 
well-ordered, 381, 620, 633 
Set builder notation, 116, 185 
Set description 
by listing its members, 116 
by the roster method, 116 
using set builder notation, 116 
Set equality, 117, 185 
Set identities, 129-132 
absorption laws, 129 
associative laws for, 129 
Cartesian product of, 123 
commutative laws for, 129 
De Morgan’s laws for, 129 
difference of, 128, 186 
distributive laws for, 129 
domination laws for, 129 
idempotent laws for, 129 
identity laws for, 129 
intersection of, 127, 185 
Set of real numbers 
cardinality of, 186 
uncountability of, 173 
Set operations, 127 
Sex, 458 
hamir, Adi, 299, 300 
hannon, Claude, 20 
hannon, Claude Elwood, 811, 812 
heffer, Henry Maurice, 34 
heffer stroke, 34, 36 
hift cipher, 295, 306 
cryptanalysis, 296 
hift ciphers 
cryptosystem, 298 
hifting, 252 
hifting index of summation, 164 
hift register, 846 
hortest-path algorithm, 709-714 
hortest-path problems, 707-716, 736 
howing two sets are equal, 121 
Sibling of vertex, 747, 803 
Sides of a polygon, 338 
Sieve of Eratosthenes, 259, 306, 560, 565 
Signature 
digital, 306 
Signatures 
digital, 303 
Signed integer, 855 
Simple circuit, 679 
Simple directed graph, 643 
Simple graphs, 642, 654-658, 735 
coloring of, 727 
connected planar, 719-723 
crossing number of, 726 
dense, 670 
edges of, 642, 663 
isomorphic, 671-675, 736 
orientation of, 740 
paths in, 679 
random, 742 
self-complementary, 677 
sparse, 670 
thickness of, 726 
vertices of, 642, 663 
with spanning trees, 785-787 
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Simple polygon, 338 
triangulation of, 338 
Simplification (rule of inference), 71 
Single-elimination tournament, 649 
Single error in an identification number, 291 
Singleton set, 118 
Sink, 901 
Six Degrees of Separation (Guare), 680 
Slackness of a job, 235 
Sloane 
Neil, 162 
Sloane, Neil, 163 
Smullyan, Raymond, 19, 20 
Snakes, poisonous, 763 
Sneakers, 300 
Soccer players, 475 
Social networks, 644 
Socks, 405 
Software 
origin of term, 11 
Software systems, 17 
Sollin’s algorithm, 803 
Solution 
optimal, 198 
Solution of a recurrence relation, 158 
Solvable problem, 232, 895, 900 
Solvable problems, 226 
Solving linear congruences, 277 
Solving satisfiability problems, 34 
Solving Sudoku puzzles, 32 
Solving systems of linear congruences, 277-279 
by back substitution, 279 
Solving using generating functions 
counting problems, 541-546 
recurrence relations, 546-548 
Sort 
binary insertion, 203 
binary insertion sort, 196 
bubble, 196, 232 
bidirectional, 233 
insertion, 196, 197, 232 
average-case complexity of, 483-484 
library, 196 
Merge, 196 
merge, 367-370, 378, 528 
complexity of, 532 
recursive, 368 
quick, 196, 371 
selection, 196, 203 
tournament, 196, 770 
Sorting, 196-198, 232 
Sorting algorithms, 196, 198 
topological, 627-629, 634 
Space, sample, 446 
Space Complexity, 232 
Space complexity, 219 
Space probe, 476 
Spam, 472-475 
Spam filters, Bayesian, 472-475 
Spanning forest, 796 
minimum, 802 
Spanning trees, 785-795, 804 
building 
by breadth-first search, 789-791 
by depth-first search, 7837-789 
definition of, 785 
degree-constrained, 806 
distance between, 797 
in IP multicasting, 786 
maximum, 802 
minimum, 797-802, 804 
rooted, 797 
Sparse graphs, 670, 802 


Sparse matrix, 670 
Specification 
of a Turing machine, 889 
Specifications 
system, 17 
Specifying 
algorithms, 192 
Spiders, Web, 794 
Spies, 112 
SQL, 588-589, 855 
Square 
perfect, 83 
Squarefree integer, 564 
Square Matrix 
diagonal of, 181 
inverse of, 184 
Square zero—one matrix 
Boolean power of, 183 
St. Petersburg Paradox, 497 
Stable assignment, 343 
Stable matching, 204 
Stable matching problem 
variations on, 234 
Standard deck, 402 
Standard deviation, 487 
Star Height, 901 
Star topology for local area network, 661 
Start symbol, 849 
State 
accepting 
finite-state automaton, 867 
final 
finite-state automaton, 867 
initial, 859 
finite-state automaton, 867 
reachable, 901 
transient, 901 
State diagram 
for finite-state machine, 860 
Strategy 
proof, 102 
Statement 
conditional, XX 
biconditional, 9 
Statement(s), A-11 
if then, 6 
procedure, A-11 
assignment, A-11—A-12 
blocks of, A-14—-A-15 
conditional, 6-9 
for program correctness, 373-375 
Statement, return, A-15 
Statements 
logically equivalent, 45 
Statement variables, 2 
States, 859 
State table 
for finite-state machine, 860 
Stephen Cook, 227 
Steroids, 475 
Stirling’s formula, 151 
Stirling, James, 151 
Stirling numbers of the first kind, 431 
signless, 443 
Stirling numbers of the second kind, 430 
Strategies 
minmax, 767, 804 
proof(, 100, 106 
Strategy 
proof, 85 
Strictly decreasing function, 143 
Strictly decreasing sequence, 403 
Strictly increasing function, 143 


Strictly increasing sequence, 403 
String, 186 
bit, 110 
empty, 186, 849 
null, 849 
recognized 
by Turing machine, 891 
String(s) 
bit, 12 
String matching, 231 
Strings, 157 
concatenation, 866 
concatenation of, 350 
counting, 388 
without consecutive 0s, 505 
decoding, 762-763 
distinguishable, 888 
generating next largest, 437 
indistinguishable, 888 
length of, 505 
recursive definition of, 350 
lexicographic ordering of, 620 
of parentheses, balanced, 382 
recognized (accepted), 868 
recursively defined, 349 
ternary, 511 
Stroke 
Sheffer, 34 
Stroke, Sheffer, 36 
Strong induction, 333-335, 378 


Strongly connected components of graphs, 686, 736 


Strongly connected graphs, 685 
Strong pseudoprime, 286 
Structural induction, 353-356, 378 
proof by, 353 
Structured query language, 588-589 
Structures, recursively defined, 349-356 
Stuarts, 151 
Subgraph, 663, 739 
Subsequence, 403 
Subset, 119, 185 
proper, 185 
Subsets 
of finite set counting, 388 
number of, 323 
sums of, 793 
Subtraction rule, 393 
Subtractors 
full, 828 
half, 828 
Subtree, 746, 748, 803 
Success, 458 
Successor 
of integer, A-5 
Successor of a set, 137 
Sudoku 
modeling as a satisfiability problem, 32 
Sudoku puzzle, 32 
Sufficient 
necessary and, 9 
Sufficient condition, 6 
expressing a conditional statement using, 6 
Sufficient for, 6 
Suitees, 204 
Suitors, 204, 343 
optimal for, 343 
Sum(s) 
Boolean, 811-812, 821, 843 
of first n positive integers, 315 
of first n positive integers, 317 
of geometric progressions, 318-319 
of subsets, 793 
Sum-of-products expansions, 820-821, 843 
simplifying, 828-840 


Summation 

index of, 163 

lower limit, 163 

notation, 162, 186 

shifting index of, 164 

upper limit, 163 
Summation formulae 

proving by mathematical induction, 315-318 
Sum of multisets, 138 
Sum of terms of a geometric progression, 164 
Sum rule, 386, 389, 439 
Sun-Tsu, 277 
Superman, 80 
Surjection, 143, 186 
Surjective (onto) function 

number of, 560-562, 566 
Surjective function, 143 
Switching circuits, hazard-free, 846 
Symbol(s), 849 

start, 849 

terminal, 849 
Symbol, Landau, 206 
Symbolic Logic (Venn), 120 
Symmetric closure of relation, 598, 634 
Symmetric difference 

of two sets, 186 
Symmetric matrix, 181, 186 
Symmetric relation, 577-578, 633 

representing 

using digraphs, 595 
using matrices, 591-592 

Syntax, 847 
System 

RSA, 299 
Systems 

hardware, 17 

software, 17 
Systems of linear congruence, 277-279 
System specifications, 17, 50 

consistent, 18 


T-shirts, 395 

Table 

truth, 4, 109 

Table(s) 

state 

for finite-state machine, 860 

Table, circular, 394 

Table membership, 186 

Tao, Terence, 263 

Tape 

Turing machine, 889 

Tautology, 25, 110 

Tee-shirts, 395 

Telephone call graph, 682 

Telephone calls, 646 

Telephone lines 
computer network with diagnostic, 642 
computer network with multiple, 642 
computer network with multiple one-way, 642 
computer network with one-way, 643 

Telephone network, 646, 682 

Telephone number, 646 

Telephone numbering plan, 387 

Template for proofs by mathematical induction, 329 

“Ten Most Wanted” numbers, 262 

Terminals, 849 

Terminal vertex, 594, 654 

Terms of a sequence 
closed formula for, 159 

Ternary search algorithm, 203 

Ternary string, 511 


Test 
Miller’s, 286 
primality, 464-465 
probabilistic primality, 464-465 
Tetromino, 109 
TeX, 208 
The Art of Computer Programming (Donald Knuth), 
196 
THE BOOK, 260 
The Elements, 260 
The Elements(Euclid), 267 
The lady or the tiger puzzle, 24 
Theorem(s), 81, 110 
alternate names for, 81 
Archimedean property, A-5 
art gallery, 735 
Bayes’, 468-475 
Bézout’s, 269, 306 
binomial, 415-418, 439 
Cantor’s, 177 
Chinese remainder, 277, 306 
Cook—Levin, 227 
Dirac’s, 701 
extended binomial, 540 
Fermat’s last, 106 
Fermat’s little, 281, 306 
four color, 728-731, 737 
fundamental, theorem of arithmetic, 258, 336-337 
Green-Tao, 263 
Hall’s marriage, 659 
handshaking, 653 
Jordan curve, 338 
Kleene’s, 880, 900 
Kuratowski’s, 724-725, 737 
Lamé’s, 347 
master, 532 
methods of proof, 82 
multinomial, 434 
Ore’s, 701, 707 
Pick’s, 342 
prime number, 262 
proving, automated, 114 
Schréder—Bernstein, 174 
Wilson’s, 285 
Theory, naive set, 118 
Theory, Ramsey, 404 
Thesis, Church—Turing, 893 
Thickness of a graph, 726 
Threshold function, 845 
Threshold gate, 845 
Threshold value, 845 
Thue, Axel, 849 
Tic-tac-toe, 766 
Tiger, 24 
Tiling(s), 103 
Tiling of checkerboard, 326 
Time complexity, 219, 232 
average-case time, 232 
average case, 220 
of algorithm for finding maximum, 219 
of linear search algorithm, 220 
worst case, 220, 232 
Tooth fairy, 112 
Top-down parsing, 853 
Topological sorting, 627-629, 634 
Topology for local area network 
hybrid, 661 
ring, 661 
star, 661 
Torus, 726 
Total expectation, law of, 492 
Total function, 152 
Totally ordered set, 619, 633 


Index I-21 


Total ordering, 619, 633 
compatible, 628, 634 
Tournament, 741 
round-robin, 468, 649 
single-elimination, 649 
Tournament sort, 196, 770 
Tower of Hanoi, 503-504 
Trace of recursive algorithm, 361, 362 
Tractable problem(s), 226, 232, 897 
Trail, 679 
Transducer(s) 
finite-state, 859 
Transformation 
affine, 296 
Transformations, 139 
Transient state, 901 
Transition function, 859 
extended, 867 
extending, 867 
extension, 867 
Transition rule(s) 
Turing machine, 889 
Transitive closure of relation, 597, 600-603, 634 
computing, 603-606 
Transitive relation, 578-580, 633 
representing, using digraphs, 596 
Transitivity law, A-2 
Translating 
English sentences to logical expressions, 16-17, 
62-63 
mathematical statements to logical expressions, 60, 
61 
nested quantifiers into English, 61-62 
logical statements into English, 61-62 
Transpose of a matrix, 181, 186 
Transposition cipher, 297 
decryption, 297 
encryption, 297 
Transposition error, 291 
“Traveler’s Dodecahedron,”’, 698, 700 
Traveling salesperson problem, 702, 709, 714-716, 
736 
Traversal of tree, 772-782, 804 
inorder, 773, 775, 778, 804 
level-order, 806 
postorder, 773, 776-778, 804 
preorder, 773-775, 778, 804 
Tree 
derivation, 899 
parse, 899 
Tree(s), 745-802 
m-ary, 748, 804 
complete, 756 
height of, 754-755 
applications of, 757-769 
as models, 749-752 
AVL, 808 
binary, 352-353, 748, 749 
extended, 352 
full, 352-353 
binary search, 757-759, 804 
binomial, 805 
caterpillar, 807 
decision, 760-762, 804 
definition of, 746 
derivation, 852-854 
extended binary, 352 
family, 745 
full m-ary, 748, 752, 804 
full binary, 352 
game, 764-769 
graceful, 807 
height-balanced, 808 
labeled, 756 
mesh of, 809 
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Tree(s)—Cont. 


parse, 852 

properties of, 752-755 

quad, 809 

rooted, 351, 747-749 
Sx-tree, 806 


B-tree of degree k, 805 
balanced, 753, 804 
binomial, 805 
decision trees, 760-762 
definition of, 803 
height of, 753, 804 
level order of vertices of, 806 
ordered, 749, 804, 806 
rooted Fibonacci, 757 
spanning, 785-795, 804 
definition of, 785 
degree-constrained, 806 
distance between, 797 
in IP multicasting, 786 
maximum, 802 
minimum, 797-802, 804 
rooted, 797 
Tree-connected network, 751-752 


Tree-connected parallel processors, 751-752 


Tree diagrams, 394-395, 439 
Tree edges, 788 
Tree traversal, 772-782, 804 
inorder, 773, 775, 778, 804 
postorder, 773, 776-778, 804 
preorder, 773-775, 778, 804 
Trial division, 258 
Triangle inequality, 108 
Triangle, Pascal’s, 419, 439 
Triangulation, 338 
Trichotomy Law, A-2 
Tricks, magic, 20 
Triomino(es), 105, 326, 333 
Right, 105, 333 
straight, 105 
Trivial proof, 84, 110 
True negative, 471 
True positive, 471 
Truth set 
of a predicate, 125 
Truth table(s), 4, 109 
for biconditional statements, 9 
for conditional statement, 6 
for conjuction, 4 
for disjunction, 4 
for exclusive or, 6 
for logical equivalences, 27 
for negation, 4 
for XOR, 6 
of compound propositions, 10 
Truth value, 3, 109 
of implication, 6 
Tukey, John Wilder, 11 
coining words, 11 
Turing, Alan Mathison, 226, 886 
Turing Award, 854 


Turing machine, 847, 886, 888, 889, 893, 899 
computing functions with, 892-893 


control unit, 889 
definition of, 889 
final state, 891 
halts, 891 


in computational complexity, 894 


initial position, 889 
initial state, 889 
nondeterministic, 893, 899 


recognition of nonregular set, 891 


sets recognized by, 891, 892 


specification, 889 
string recognition by, 891 
tape, 889 
transition rules, 889 
types of, 893 
Twin prime conjecture, 264 
Twin primes, 264 
Two’s complement representations, 256 
Two-dimensional array, 662-663 
Two children problem, 498 
Type, 117 
Type 0 grammar, 851, 899 
Type | grammar, 851, 899 
Type 2 grammar, 851, 899 
Type 3 grammar, 851, 883, 899 


U.S. Coast Survey, 38 
Ulam’s problem, 536 
Ulam, Stanislaw, 536 
Ulam numbers, 188 
Unary representations, 892 
Uncomputable function, 175, 186, 896, 900 
Uncountability 

set of real numbers, 173 
ncountable set, 186 

decidable problem, 895 
ndefined values 

of partial function, 152 
derlying undirected graph, 654, 736 
ndirected edges, 644, 735 

of simple graph, 643 
directed graphs, 644, 653, 735 
connectedness in, 681-685 
Euler circuit of, 694 

Euler path of, 698 

orientation of, 740 

paths in, 679, 736 

underlying, 654, 736 
Unicasting, 786 
Unicorns, 112 
Unicycle, 812 
Uniform distribution, 454, 494 
Unimodal sequence, 568 
Union 

of graphs, 664 

of three finite sets, number of elements in, 

554-556, 566 

of two finite sets, number of elements in, 553, 565 
inion of 

fuzzy sets, 138 

nion of a collection of sets, 133 
nion of multisets, 138 
nion of sets, 127, 185 
niqueness proof, 110 
niqueness quantifier, 44 
niquness 

proof, 99 
nit (Egyptian) fraction, 380 
nit-delay machine, 861-862 
nit-delay machine, 861 

nited States Coast Survey, 38 

nit property, in Boolean algebra, 815 
INIVAC, 872 

niveral quantification, 110 
niversal address system, 772-773 
niversal generalization, 76 
niversal instantiation, 75 
niversal modus ponens, 77 
niversal modus tollens, 77 
niversal product code (UPC), 290 
niversal quantification, 40 
niversal quantifier, 40 
niversal set, 118, 185 
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niverse of discourse, 40, 110 

labeled 

boxes, 428 

objects, 428 

less, 6 

expressing conditional statement using, 7 
nsatisfiable compound proposition, 30 
nsolvable problem, 232, 895, 900 
nsolvable problems, 226 
PC check digit, 290 
pper bound, A-2 

of lattice, 637 

of poset, 625, 634 
pper limit of a summation, 163 
pper triangular matrix, 231 
tilities-and-houses problem, 718 
zbekistan, 192 
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Vacuous proof, 84, 110 
Valid argument, 69, 110 
Valid argument form, 70, 110 
Valid partner, 234 
Value 
truth, 3, 109 
Value(s) 
expected, 477-480, 491, 494 
in hatcheck problem, 495 
linearity of, 477-484, 494 
of inversions in permutation, 482-484 
final, A-14 
initial, A-14 
of tree, 767 
of vertex in game tree, 767-769, 804 
threshold, 845 
Vandermonde’s identity, 420-421 
Vandermonde, Alexandre-Théophile, 420 
Variable(s) 
Boolean, 11, 110, 814, 820, 843 
bound, 44, 110 
free, 44, 110 
propositional, 2, 109 
random, 454, 460-461 
covariance of, 494 
definition of, 494 
distribution of, 460, 494 
expected values of geometric, 477-480 
geometric, 484-485 
geometric expected values of, 491, 494 
independent, 485-487, 494 
indicator, 492 
standard deviation of, 487 
variance of, 477-480, 494 
statement, 2 
Variance, 477, 487-490, 494 
Veitch, E. W., 830 
Vending machine, 858-859 
Venn, John, 118, 120 
Venn diagrams, 118, 185 
Verb, 848 
Verb phrase, 848 
Vertex (vertices), 594 
adjacent, 651, 654, 735 
ancestor of, 747, 803 
basis, 691 
child of, 747, 749, 803 
connecting, 651 
connectivity, 684 
counting paths between, 688-689 
cut, 683, 684 
degree of, 652 
degree of, in undirected graph, 652 
descendant of, 747, 804 
distance between, 741 
eccentricity of, 757 
end, 654 


Vertex (vertices) —Cont. 
in-degree of, 654, 736 
independent set of, 741 
initial, 594, 654 
interior, 603 
internal, 748, 804 
isolated, 652, 736 
level of, 753, 804 
level order of, 806 
number of, of full binary tree, 355 
of directed graph, 643, 654 
of directed multigraph, 644 
of multigraph, 642 
of polygon, 338 
of pseudograph, 643 
of simple graph, 642, 668 
of undirected graph, 644, 652, 654 
out-degree of, 654, 736 
parent of, 747, 803 
pendant, 652, 736 
sibling of, 747, 803 
terminal, 594, 654 
value of, in game tree, 767-769, 804 
Vertex set, 338 
bipartition of, 656 
Very large scale integration (VLSI graphs, 744 
Vigenére cipher, 304 
cryptanalysis, 305 
Vocabulary, 849, 899 
“Voyage Around the World” Puzzle, 700 


Walk, 679 
closed, 679 
Wall Street, 198 
Warshall’s algorithm, 603-606 
Warshall, Stephen, 603, 604 


WAVES, Navy, 872 
Weakly connected graphs, 686 
Web crawlers, 646 
Web graph, 646-647, 794 
strongly connected components of, 686 
Web page(s), 646, 686, 794 
searching, 18 
Web spiders, 794 
Weighted graphs, 736 
minimum spanning tree for, 798-802 
shortest path between, 707-714 
traveling salesperson problem with, 714-716 
Well-formed expressions, 354 
Well-formed formula, 350 
for compound propositions, 354 
in prefix notation, 784 
of operators and operands, 351 
structural induction and, 354 
Well-founded induction, principle of, 635 
Well-founded poset, 632 
Well-ordered induction, 634 
principle of, 620 
Well-ordered set, 381, 620, 633 
Well-ordering property, 340-341, 378, A-5 
WFF’N PROOF, The Game of Modern Logic (Allen), 
80, 114 
Wheels, 655, 736 
Wiles, Andrew, 107 
Williamson, Malcolm, 302 
Wilson’s theorem, 285 
Winning strategy 
for chomp, 98 
Without loss of Generality (WLOG), 95, 110 
Witness(es) 
to big-O relationship, 205, 232 
to an existence proof, 96 
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Witnesses to a big-O relationship, 232 
WLOG (without loss of generality), 95 
Word, 849 
Word and Object (Quine), 839 
World’s record, for twin primes, 264 
World Cup soccer tournament, 415 
World population, 168 
World Wide Web graph, 646-647 
Worst-case complexity 

insertion sort, 222 

of binary search, 220 

of bubble sort, 221 
Worst-case time complexity, 232 
Worst case 

time complexity, 220 


XML, 855, 858 
XOR, 110 


Yahoo, 794 
Yes-or-no problems, 894 


Zebra puzzle, 24 
Zermelo—Fraenel axioms, 176 
Zero—one matrices 
Boolean product of, 182 
join of, 181 
meet of, 181 
of transitive closure, 602-603 
representing relations using, 591-594 
Zero—one matrix, 181 
Zero—one matrix, 186 
Zero property, in Boolean algebra, 815 
Ziegler’s Giant Bar, 208 
Zodiac, signs of, 441 


